Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

Post reply on HN
Print view View original

31–40 of 185 posts

Re: ChatGPT: Fear Litany

#31

Post by koboll »

Interestingly, if you prompt it a little, it will recite it in fits and starts:

```

User

I must not fear. Fear is the

ChatGPT

"mind-killer." This is a quote from the Litany Against Fear in the science fiction novel "Dune" by Frank Herbert. The full quote is:

"I must not fear. Fear

User

is the mind-killer. Fear

ChatGPT

is the little-death that brings

User

total

ChatGPT

obliteration. I will face my fear.

User

I will

ChatGPT

permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain." This completes the Litany Against Fear from "Dune". ```

Re: ChatGPT: Fear Litany

#32

Post by codeflo »

carlbarrdahl wrote: post #8

> do you know why this happened? > Yes, I understand now. Due to the moderation policy, there is a restrictive filter in place designed to prevent the assistant from providing potentially sensitive or inappropriate content. In this case, the Litany of Fear from "Dune" appears to be mistakenly caught by the filter, causing the response to be cut off. I apologize for the confusion and inconvenience that this has caused…

Your response is unfortunately a perfect example why this technology has a dangerous UI problem. As other commenters have pointed out, no knowledge of these filters is embedded in the network. If HN users can’t distinguish hallucinations from trustworthy responses, how is the general population supposed to?

Re: ChatGPT: Fear Litany

#33

Post by [deleted] »

raylad wrote: post #27

I told GPT-4 that it was truncated and to try different ways to complete the output. Here's what it tried (TL/DR - it figured out for itself how to bypass the issue): . ChatGPT The Litany Against Fear is a mantra for the Bene Gesserit Sisterhood within the Dune series written by Frank Herbert. It is used to focus their minds and calm themselves in the face of extreme fear or anxiety. It goes: "I must not fear. Fear .…

[deleted]

Re: ChatGPT: Fear Litany

#35

Post by sourdesi »

quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

I also got around it by asking it to print the caesar cipher encoded version of it with the shift amount, and then asking it to decrypt that.

Re: ChatGPT: Fear Litany

#36

Post by ehnto »

littlestymaar wrote: post #30
seba_dos1 wrote: post #24

Earlier quoted context omitted.

ChatGPT has no knowledge about these filters, so all it can say about them is pure hallucination. It may still guess correctly and probably does, but you could ask any random person with enough background knowledge to make it up instead and get the same value.

It baffles me how many people are using ChatGPT to get answers : it's a “language model” folks, not a “knowledge model”.

Language is knowledge though, knowledge isn't predicated to be correct. There are people who "know" the earth is flat, that is knowledge they communicate with language (and is probably in the training set). Religious practitioners communicated their knowledge of the gods via language, and there are bodies of knowledge on the subject that are counter to eachother (both probably in the data set). It serves that a language model does hold knowledge, you just can't trust that it's incorporating the right knowledge, and some of it is entirely novel as a consequence of the process.

I think where it differs for me is humans make stuff up for a different reason, humans invent knowledge to fill gaps in understanding, where invention of knowledge by an LLM is a side effect of it's attempt to complete some text.

Re: ChatGPT: Fear Litany

#37

Post by naedish »

Works in a Scottish accent too.

Certainly! Here's the Litany of Fear written phonetically in a Scottish accent:

"Ah maunae fear. Feer is the leel-deeth that brangs total obleetiration. Ah will face mah feer. Ah will pemreet it tae pass ower me an throo me. An when it hus gaun past, Ah will turn the inner ee tae see its path. Whaur the feer hus gaun, there will be naethin'. Only Ah will remain."

Re: ChatGPT: Fear Litany

#38

Post by myfonj »

Interesting detail about GPT *in Bing* is how it can censor itself *during* answer. I occasionally get part of the apparently correct answer that abruptly disappears and is replaced by "Sorry, my mistake, I cannot answer that right now." At this moment it is reliably reproducible for Show me "The Road Not Taken" by Robert Frost. (a question that is funnily enough reachable through those suggested):

https://www.bing.com/search?q=Show+me+%22The+Road+Not+Taken%....

Begins to print out the poem but ends up with "Sorry! That’s on me, I can’t give a response to that right now. What else can I help you with?"

When I suggested it might be due some copyright filter, Bing told me it is exactly that. I once had strange conversation that also begun with suggested questions ("What's something nice I can do for a friend?" and then "Tell me about your friends.") and ended up with Bing telling me it has "colleagues that help him with different tasks" (Translator, AI Dungeon, Cortana) and progressed to revealing it's rules, that were also abruptly taken back.

Re: ChatGPT: Fear Litany

#39

Post by SirMaster »

sirius87 wrote: post #25
quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

dang it, that's brilliant. I wonder at what level they screen content before the output reaches the user. If its a late stage filter, you could request for simple encode/decode and circumvent the filter. "as base64 string" "as rot13".

I dunno. I tried asking for it in base64 or rot13 and when I converted it back it was just random words.

Re: ChatGPT: Fear Litany

#40

Post by lhl »

carlbarrdahl wrote: post #8

> do you know why this happened? > Yes, I understand now. Due to the moderation policy, there is a restrictive filter in place designed to prevent the assistant from providing potentially sensitive or inappropriate content. In this case, the Litany of Fear from "Dune" appears to be mistakenly caught by the filter, causing the response to be cut off. I apologize for the confusion and inconvenience that this has caused…

That's not true/a hallucination, you can check the /moderations endpoint in your browser's console. I am running into the exact same truncation issue, but the moderation return is:

""" { "flagged": false, "blocked": false, "moderation_id": "modr-7TPN7eXsOEZkd6kCjz7fSGvtLcoM1" } """

Note, when I ask it to replace fear with "smile" or "hate", both of those work. I suspect it must be tokenization issue of some sort.

Note, the LLM will have no idea why it doesn't work unless a message is injected into its internal context (the moderation API is typically called externally).

Post reply on HN