Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

Post reply on HN
Print view View original

131–140 of 185 posts

Re: ChatGPT: Fear Litany

#131

Post by masklinn »

Sharlin wrote: post #127
masklinn wrote: post #101

Earlier quoted context omitted.

Sure but nobody knows the word “confabulation”, and lying / making things up implies intent. So “hallucination” hews close enough to have good explanatory powers.

Confabulation is unintentional, FWIW: > In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. […] Confabulation occurs when individuals mistakenly recall false information, without intending to deceive.

Yes, which is why I agree that it’s a better term. That’s not the issue.

Re: ChatGPT: Fear Litany

#132

Post by ehnto »

nirse wrote: post #66
ehnto wrote: post #36

Earlier quoted context omitted.

Language is knowledge though, knowledge isn't predicated to be correct. There are people who "know" the earth is flat, that is knowledge they communicate with language (and is probably in the training set). Religious practitioners communicated their knowledge of the gods via language, and there are bodies of knowledge on the subject that are counter to eachother (both probably in the data set). It serves that a langu…

In philosophy, the most common definition of knowledge is that it is true, justified belief[1]. In that definition knowledge certainly is predicated to be correct. [1] https://plato.stanford.edu/entries/knowledge-analysis

So is an argument that an LLM has no knowledge because it can't reason about justification of it's replies? I could see that. A lot of justification is just "because I read it in a book I trust" or "heard it from an authority" which isn't all that dissimilar to an LLM's outcome. An LLMs response is also justified by it's training, not unlike a person in a position of authority.

It doesn't challenge the idea that both a religious believer and non-believer have a certain knowledge of the world. These kinds of conflicts of knowledge are common, not just in religion, there is no single understanding of the world to reference against.

Re: ChatGPT: Fear Litany

#133

Post by masklinn »

est wrote: post #118
masklinn wrote: post #63

Earlier quoted context omitted.

> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it a legit concept or just "the dog ate my homework" type of excuse for anything? It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens. So essentially it always hallucinates (makes shi…

> it just adds tokens probabilistically based on the previous tokens I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

> I mean, isn't this what humans do all the time?

No? Most humans don’t randomly vomit text based on what sounds good.

> Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

Which shows a much higher level of understanding, both of the field (which may be flawed), and of their own understanding of the field (which they point out).

An LLM does not to that, it doesn’t just repeat potentially wrong hearsay or incorrect memories (let alone having actual understanding and knowledge of the field), it confidently writes out delusions.

Re: ChatGPT: Fear Litany

#134

Post by jncfhnb »

Sharlin wrote: post #58
d13 wrote: post #46

Earlier quoted context omitted.

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

Uh, obviously it can't recite the entirety of Dune, for example. But just as obviously, the Litany Against Fear is famous enough to likely exist verbatim thousands of times in its training corpus.

I don’t think this is obvious at all. Ignore the algorithm and temperature and all that for a bit and just have it show you the word probabilities. I wouldn’t bat an eye if the entirety of dune was in there.

Re: ChatGPT: Fear Litany

#135

Post by seba_dos1 »

Sharlin wrote: post #106
littlestymaar wrote: post #87

Earlier quoted context omitted.

> So, when it helped me solve a math problem by noticing a trick that I had missed – which was clearly the correct thing to do in retrospect – was I somehow in the wrong to ask it for help? There's nothing wrong in using LLMs. > Getting answers out of it is absolutely a reasonable thing to do. Blindly trusting those answers without verification, that's another thing entirely. We're basically saying the same thing but…

Fair enough. Although of course you shouldn't blindly trust any source; it's really about degrees of trustworthiness.

Of course. It's just that ChatGPT's degree of trustworthiness is near zero, so citing it is pretty much worthless - unlike citing someone considered an expert in their field, for example.

It may still be a very useful tool, but not when used this way.

Re: ChatGPT: Fear Litany

#136

Post by dacryn »

d13 wrote: post #46
quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

at some points it starts to resemble more of a lossy compression algorithm. The 'next most likely word' aspect of it all seem to encode entire sequences of text. Especially if it is a bit of a niche domain. Then these texts can be 'retrieved' from the 'compressed' set of examples.

Re: ChatGPT: Fear Litany

#137

Post by number6 »

est wrote: post #118
masklinn wrote: post #63

Earlier quoted context omitted.

> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it a legit concept or just "the dog ate my homework" type of excuse for anything? It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens. So essentially it always hallucinates (makes shi…

> it just adds tokens probabilistically based on the previous tokens I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

"As a Language Model" is the new "I am not a lawyer"

Re: ChatGPT: Fear Litany

#138

Post by jncfhnb »

I think it’s much more interesting that the user can write “brah” and the model can interpret that as “you are continuing to make the same mistake and it’s dumb” and then follow suit with a weird role play of trying to hack itself.

Re: ChatGPT: Fear Litany

#139

Post by Sharlin »

masklinn wrote: post #131
Sharlin wrote: post #127

Earlier quoted context omitted.

Confabulation is unintentional, FWIW: > In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. […] Confabulation occurs when individuals mistakenly recall false information, without intending to deceive.

Yes, which is why I agree that it’s a better term. That’s not the issue.

Ah, I misinterpreted your previous comment!

Re: ChatGPT: Fear Litany

#140

Post by remuskaos »

I've just tried it, and for me it displays the entire litany (afaict):

Certainly, the Litany Against Fear is a well-known mantra in Frank Herbert's Dune series. It goes as follows:

"I must not fear. Fear

is the mind-killer. Fear

is the little-death that brings

total obliteration. I will

face my fear. I will permit

it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain."

https://chat.openai.com/share/6578d97b-fe61-4dd8-8865-99e20c...

---

But you can see the formatting is a little off. In fact, each time there was a double new line, the output paused for several seconds, before continueing. So what someone in a sibling comment mentioned about a content filter seems plausible to me.

Post reply on HN