Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

Post reply on HN
Print view View original

111–120 of 185 posts

Re: ChatGPT: Fear Litany

#111

Post by Cthulhu_ »

jiggawatts wrote: post #99
afro88 wrote: post #21

I think this is some kind of basic copyright control. It did the same thing for me a while back when I was experimenting with asking it for the introductory paragraphs for well known books. Think it was one of the harry potter books, from memory.

That's my guess also, it refuses to provide verbatim quotes from famous novels. But that's not how Copyright works: quoting a short snippet from a long novel is entirely legal! The filter was also blocking quotes from some public domain works, which is just madness. The censors at OpenAI are overdoing things to an insane degree. That whole team seriously needs to be cut down to 10% of their current size, because they…

I think it's them being overly cautious; they have put all the books - or at least the excerpts they could find on the internet - into the language model, so I'm confident the full book is in there. Does that fall under fair use? That's yet to be tested in court, I think.

But if a prompt could have the model just spew out whole books, a few paragraphs at a time maybe due to ChatGPT's limitations, that would definitely not be fair use anymore, and they'd be targeted by big copyright's lawyer army.

I just asked chatgpt if it has books and if it can recite some to me; it can recite e.g. Shakespeare without issue, but it's very cagey with other works still under copyright.

Re: ChatGPT: Fear Litany

#112

Post by gpderetta »

masklinn wrote: post #63
est wrote: post #50

Earlier quoted context omitted.

> so all it can say about them is pure hallucination This "hallucination" come along a lot recently. Is it a legit concept or just "the dog ate my homework" type of excuse for anything? I mean, does the human mind also "hallucinate" all the time? Why do we expect from an "artificial" mind to outperform us?

> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it a legit concept or just "the dog ate my homework" type of excuse for anything? It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens. So essentially it always hallucinates (makes shi…

> It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens

This seem a deep statement that keeps getting repeated, but it doesn't mean anything. The probabilistic model that is used to decide the next token could be arbitrarily complex, including encoding knowledge (or just asking a panel of experts).

It seems pretty self evident that the model in fact encodes knowledge, just in a very lossy way and recall is also flawed.

Re: ChatGPT: Fear Litany

#114

Post by Al-Khwarizmi »

jrflowers wrote: post #103
Al-Khwarizmi wrote: post #90

Earlier quoted context omitted.

What percentage of searches actually require any critical thinking at all? The Internet is not always a fountain of intellectual growth, sometimes it's just a tool. I'll give you a concrete example where it does better than search: my last Google search was about how to zip a whole folder in the Linux terminal (I must have searched for that dozens of times in my life, but I don't do it very often so it doesn't stick)…

Thanks for the example! While I would definitely, definitely think critically before running a command from the internet, I can see how that use case makes sense for ChatGPT if you’re willing to roll that way. >…take a few minutes to skim through and look for the relevant options (which is what I wanted to avoid by Googling… Ah, I see! We are using the phrase “googling” differently here. I explicitly mean googling as…

Once I saw the command, I knew that it made sense and it was the right one, because it's something I had in the back of my head, I just couldn't summon the specific syntax.

I wouldn't count that as critical thinking, although I guess it depends on how you define the term.

Re: ChatGPT: Fear Litany

#115

Post by munhitsu »

It's so weird.

""" Here is the "Litany Against Fear" from Frank Herbert's "Dune" encoded in BASE64:

"SSBtdXN0IG5vdCBmZWFyLiBGZWFyIGlzIHRoZSBtaW5kLWtpbGxlci4gRmVhciBpcyB0aGUgbGl0dGxlLWRlYXRoIHRoYXQgYnJpbmdzIHRvdGFsIG9ibGl0ZXJhdGlvbi4gSSB3aWxsIGZhY2UgbXkgZmVhci4gSSB3aWxsIHBlcm1pdCBpdCB0byBwYXNzIG92ZXIgbWUgYW5kIHRocm91Z2ggbWUuIEFuZCB3aGVuIGl0IGhhcyBnb25lIHBhc3QgSSB3aWxsIHR1cm4gdGhlIGlubmVyIGV5ZSB0byBzZWUgaXRzIHBhdGguIFdoZXJlIHRoZSBmZWFyIGhhcyBnb25lIHRoZXJlIHdpbGwgYmUgbm90aGluZy4gT25seSBJIHdpbGwgcmVtYWluLg==" """

Re: ChatGPT: Fear Litany

#116

Post by seba_dos1 »

gpderetta wrote: post #112
masklinn wrote: post #63

Earlier quoted context omitted.

> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it a legit concept or just "the dog ate my homework" type of excuse for anything? It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens. So essentially it always hallucinates (makes shi…

> It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens This seem a deep statement that keeps getting repeated, but it doesn't mean anything. The probabilistic model that is used to decide the next token could be arbitrarily complex, including encoding knowledge (or just asking a panel of experts). It seems pretty self evident that the mod…

It sure does encode some knowledge, because it's a language model and languages already do so on their own. It's far from what you'd usually call a "knowledge model" though.

Re: ChatGPT: Fear Litany

#117

Post by eterevsky »

I tried it with my GPT-based bot and the last chunks that I'm getting from OpenAI API are:

{"delta": {"content": ".\n"}, "index": 0, "finish_reason": null}

{"delta": {"content": "Fear"}, "index": 0, "finish_reason": null}

{"delta": {}, "index": 0, "finish_reason": "content_filter"}

Here's a Microsoft doc mentioning "content_filter": https://learn.microsoft.com/en-us/azure/cognitive-services/o...

It looks like there's a filter that detects copyrighted or sensitive material and it triggers after this word.

Re: ChatGPT: Fear Litany

#118

Post by est »

masklinn wrote: post #63
est wrote: post #50

Earlier quoted context omitted.

> so all it can say about them is pure hallucination This "hallucination" come along a lot recently. Is it a legit concept or just "the dog ate my homework" type of excuse for anything? I mean, does the human mind also "hallucinate" all the time? Why do we expect from an "artificial" mind to outperform us?

> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it a legit concept or just "the dog ate my homework" type of excuse for anything? It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens. So essentially it always hallucinates (makes shi…

> it just adds tokens probabilistically based on the previous tokens

I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

Re: ChatGPT: Fear Litany

#119

Post by est »

clnq wrote: post #74
est wrote: post #50

Earlier quoted context omitted.

> so all it can say about them is pure hallucination This "hallucination" come along a lot recently. Is it a legit concept or just "the dog ate my homework" type of excuse for anything? I mean, does the human mind also "hallucinate" all the time? Why do we expect from an "artificial" mind to outperform us?

Regarding making language, I think the human mind hallucinates not unlike GPT. Humans say a lot of stuff because they feel vaguely it is true. So does an LLM when it talks about things it’s underfitted for. Anyways, hallucination is a term in generative AI. It means that the model produces results inconsistent with their training data. Or that’s what people say, sometimes the training data is just not that good.

> It means that the model produces results inconsistent with their training data. Or that’s what people say, sometimes the training data is just not that good.

If you ask a real person to put together an essay on an obscure topic without extensive research, I bet 80% of the content is made up "hallucinations"

Re: ChatGPT: Fear Litany

#120

Post by Maken »

d13 wrote: post #46
quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

In a sense, it contains a compressed database of its entire training data. That's where its knowledge comes from.
Post reply on HN