Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

Post reply on HN
Print view View original

181–185 of 185 posts

Re: ChatGPT: Fear Litany

#181

Post by riezebos »

quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

Translating it to Dutch and back changes a few words, apparantely that also passes through the filter:

https://archive.is/RztYU

Re: ChatGPT: Fear Litany

#182

Post by seba_dos1 »

est wrote: post #145
masklinn wrote: post #133

Earlier quoted context omitted.

> I mean, isn't this what humans do all the time? No? Most humans don’t randomly vomit text based on what sounds good. > Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff. Which shows a much higher level of understanding, both of the field (which may be flawed), and of their own understanding of the field (which they point out). An LLM does not to…

> Most humans don’t randomly vomit text based on what sounds good. Unless humans were given a task? e.g. taking exams while un-prepared. My kid usually gives me a long description of imaginary stuff based on the name only or brief intro. It's very fun when finally the real deal was revealed.

That's absolutely right. That said, people don't usually take exam output of unprepared students and expect it to be useful :)

Re: ChatGPT: Fear Litany

#183

Post by utucuro »

pclmulqdq wrote: post #12
flockonus wrote: post #5

Hard to believe how it has "self understanding" to bypass its own bug (or whatever filtering mechanism) is preventing it to answer.

It's probably bypassing a copyright filter, not a bug.

I'm not sure if it is a copyright filter at this point. After numerous simple tries to get it to recite the litany, I asked for the inscription on the one ring, just as straightforward. No hesitation, no wait, recited the whole thing, formatted in two lines and then went on to interpret what it meant. A copyright filter should have prevented that too, I think, if the length had been the issue, it would not always stop at "Fear" on the second line...

Re: ChatGPT: Fear Litany

#184

Post by jncfhnb »

Sharlin wrote: post #146
jncfhnb wrote: post #134

Earlier quoted context omitted.

I don’t think this is obvious at all. Ignore the algorithm and temperature and all that for a bit and just have it show you the word probabilities. I wouldn’t bat an eye if the entirety of dune was in there.

Maybe obvious was too strong a word, but I very much doubt that it contains a lossless representation of any copyrighted novel-length work. Just due to the pigeonhole principle if not anything else. The model is a vastly compressed approximation of the full training corpus.

The intersection space of 10^15 parameters is a lot of pigeon holes though.

Re: ChatGPT: Fear Litany

#185

Post by dwaltrip »

littlestymaar wrote: post #175
dwaltrip wrote: post #165

Earlier quoted context omitted.

Its trustworthiness is not near zero… There are many categories and types of questions where you can have a good degree of confidence in the answer. This is especially true for questions where a rough approximation is acceptable. Off the top of my head: basic science questions, simple programming questions, and widely known historical facts. If you expect a topic to be very well-represented in the training data, then…

> Its trustworthiness is not near zero It is, because it can give you correct answers as well as entirely stupid ones (like the infamous “horse eggs” example of a few month ago). > basic science questions, For math it's pretty much hit or miss, even for straightforward stuff (and I'm not talking about doing calculation) > and widely known historical facts. This is really something you should not use ChatGPT for, beca…

My experience has differed from yours then.

I think you overstate the frequency of which it hallucinates for simpler questions. I am usually using GPT-4, which is more reliable. For the types of questions I’m talking about, I’d say GPT-4 is at least 90% accurate.

Often times, I’m just looking something up due to curiosity while discussing with friends. It’s replacing a google search. The failure mode is that there is small chance I’m passing along incorrect information to my friends. Not a big deal.

If 90% accuracy is not good enough or if the question is not simple, then I’ll take more care.

Perhaps we are also getting into a semantic debates about what “trustiworthiness” means…

Post reply on HN