Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

51–60 of 185 posts

Re: ChatGPT: Fear Litany

#51

Earlier quoted context omitted.

ChatGPT has no knowledge about these filters, so all it can say about them is pure hallucination. It may still guess correctly and probably does, but you could ask any random person with enough background knowledge to make it up instead and get the same value.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

Not like humans are much different, especially those I have immediate access to.

Re: ChatGPT: Fear Litany

#53

Earlier quoted context omitted.

ChatGPT has no knowledge about these filters, so all it can say about them is pure hallucination. It may still guess correctly and probably does, but you could ask any random person with enough background knowledge to make it up instead and get the same value.

Heh, when I ask why it's cut off, this is the response: >The partial completion of the Litany of Fear in the previous response occurred because of the model's tendency to generate text in chunks or segments, typically up to a certain token limit. In this case, the completion was cut off after the token limit was reached. GPT-3.5 has a maximum token limit of 4096 tokens, and if the response reaches that limit, it may…

This response is certainly incorrect: had the token limit really be reached, no more text could come out. Not the Litany of Fear, nor this explanation.

Re: ChatGPT: Fear Litany

#54
post #36

Earlier quoted context omitted.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

Language is knowledge though, knowledge isn't predicated to be correct. There are people who "know" the earth is flat, that is knowledge they communicate with language (and is probably in the training set). Religious practitioners communicated their knowledge of the gods via language, and there are bodies of knowledge on the subject that are counter to eachother (both probably in the data set). It serves that a langu…

Language is mostly lies.

Re: ChatGPT: Fear Litany

#55

Earlier quoted context omitted.

ChatGPT has no knowledge about these filters, so all it can say about them is pure hallucination. It may still guess correctly and probably does, but you could ask any random person with enough background knowledge to make it up instead and get the same value.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

There is a group of people that are ardent in the conviction that if we wait long enough and wish hard enough that ChatGPT will replace the concept of googling things.

I’m personally open to that being the case with some LLM someday, but there’s a lot of people ideologically and financially invested in that perception being accurate today and applying to ChatGPT in particular.

People hate it when you point out that they’ve confused the modern equivalent of a Speak and Spell with The Overmind.

Re: ChatGPT: Fear Litany

#56
post #36

Earlier quoted context omitted.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

Language is knowledge though, knowledge isn't predicated to be correct. There are people who "know" the earth is flat, that is knowledge they communicate with language (and is probably in the training set). Religious practitioners communicated their knowledge of the gods via language, and there are bodies of knowledge on the subject that are counter to eachother (both probably in the data set). It serves that a langu…

> knowledge isn't predicated to be correct.

It kind of is, though. The field of philosophy called epistemology deals with what knowledge is and how it is obtained, and the field overwhelmingly agrees that it is necessary (but not sufficient) for knowledge to be a (1) belief that is (2) true, and (3) justified. There is some disagreement when it comes to justification, and a lot of nuance in whether every justified true belief is knowledge, but pretty much no disagreement that knowledge must be a thing that is true and involve the mental state of believing that the thing is true.

Language doesn't imply belief (1), and it doesn't imply justification (2), and it certainly doesn't imply truth (3). Language is a medium in which knowledge can be expressed, but not knowledge itself (one can of course have knowledge about language, but that is not the same thing). Language is also a medium in which things that aren't knowledge can be expressed, such as falsehoods, nonsense statements, paradoxes, etc. LLMs generate language without belief, they sometimes do generate statements that are true and justified, but definitely not always.

I think the terms "data" or "information" are better than "knowledge" for what LLMs are trained on and produce.

(1) "The sky is green." Lying in general, are ways to use language to express something that isn't a belief. Also imperatives like "Please take out the garbage" or questions like "Did you take out the garbage?" don't seem to express a belief at all

(2) "There is a planet somewhere in the universe where s'mores naturally occur without intentional assembly by intelligent beings." This might be true, and I might believe it, but there is no justification for it. Still expressed in language. Questions and imperatives come up here too.

(3) See (1), or any other case of lying or being mistaken.

Re: ChatGPT: Fear Litany

#57
post #50

Earlier quoted context omitted.

ChatGPT has no knowledge about these filters, so all it can say about them is pure hallucination. It may still guess correctly and probably does, but you could ask any random person with enough background knowledge to make it up instead and get the same value.

> so all it can say about them is pure hallucination This "hallucination" come along a lot recently. Is it a legit concept or just "the dog ate my homework" type of excuse for anything? I mean, does the human mind also "hallucinate" all the time? Why do we expect from an "artificial" mind to outperform us?

It "came along a lot recently" because everything ChatGPT is "recently" -- it only came out 6 months ago.

Re: ChatGPT: Fear Litany

#58
post #46

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

Uh, obviously it can't recite the entirety of Dune, for example. But just as obviously, the Litany Against Fear is famous enough to likely exist verbatim thousands of times in its training corpus.

Re: ChatGPT: Fear Litany

#59
post #46

Earlier quoted context omitted.

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

I thought Wikipedia was included in its training corpus, in which case the full Litany of fear is available on Wikipedia. https://en.m.wikipedia.org/wiki/Bene_Gesserit

And thousands of other websites…

Re: ChatGPT: Fear Litany

#60
post #36

Earlier quoted context omitted.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

Language is knowledge though, knowledge isn't predicated to be correct. There are people who "know" the earth is flat, that is knowledge they communicate with language (and is probably in the training set). Religious practitioners communicated their knowledge of the gods via language, and there are bodies of knowledge on the subject that are counter to eachother (both probably in the data set). It serves that a langu…

Knowledge not necessarily being correct, and language being the same thing as knowledge are big ideas! See for example:

https://en.wikipedia.org/wiki/Gettier_problem

https://en.wikipedia.org/wiki/Philosophical_Investigations

Post reply on HN