Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT: Fear Litany

web.archive.org

Post reply on HN
Print view View original

171–180 of 185 posts

Re: ChatGPT: Fear Litany

#171

Post by Al-Khwarizmi »

jrflowers wrote: post #168
Al-Khwarizmi wrote: post #110

Earlier quoted context omitted.

Yes, I also got that first result that (from the snippet) seems to go straight to the command, but the funny thing is that the link doesn't work, at least from my end. And the next ones are all lists of commands that would take a nontrivial amount of time to skim, as I mentioned.

I don’t follow. Both ChatGPT and Google gave you the same command, but Google’s response contained links, therefore Google’s response is less good. Wouldn’t you have had an identical experience copy and pasting from Google as you did ChatGPT? Did you recognize the correct command immediately in ChatGPT but have difficulty recognizing that same command on Google’s page?

The Google snippet doesn't have the full syntax, it mentions the "-r" option but doesn't reach an usage example. That's presumably in the link that didn't worn for me.

Re: ChatGPT: Fear Litany

#172

Post by voxelghost »

Sharlin wrote: post #146
jncfhnb wrote: post #134

Earlier quoted context omitted.

I don’t think this is obvious at all. Ignore the algorithm and temperature and all that for a bit and just have it show you the word probabilities. I wouldn’t bat an eye if the entirety of dune was in there.

Maybe obvious was too strong a word, but I very much doubt that it contains a lossless representation of any copyrighted novel-length work. Just due to the pigeonhole principle if not anything else. The model is a vastly compressed approximation of the full training corpus.

Even though vastly compressed, in terms of lossy vs. fidelity, it seems to be able to recite pretty much any bible verse you ask it for.

Re: ChatGPT: Fear Litany

#173

Post by aerio »

JimDabell wrote: post #130
aerio wrote: post #78

Earlier quoted context omitted.

That sounds like it was trained on a broke training set, such as the Scots Wikipedia: https://www.theregister.com/2020/08/26/scots_wikipedia_fake/

They asked for it to be written phonetically in a Scottish accent; they didn’t ask for it to be written in Scots.

How do you know? You can't see the prompt.

There's also not a huge amount of resources for "Scottish written phonetically" except the Scots Wikipedia which was exactly that and written by an American kid.

Re: ChatGPT: Fear Litany

#175

Post by littlestymaar »

dwaltrip wrote: post #165
seba_dos1 wrote: post #135

Earlier quoted context omitted.

Of course. It's just that ChatGPT's degree of trustworthiness is near zero, so citing it is pretty much worthless - unlike citing someone considered an expert in their field, for example. It may still be a very useful tool, but not when used this way.

Its trustworthiness is not near zero… There are many categories and types of questions where you can have a good degree of confidence in the answer. This is especially true for questions where a rough approximation is acceptable. Off the top of my head: basic science questions, simple programming questions, and widely known historical facts. If you expect a topic to be very well-represented in the training data, then…

> Its trustworthiness is not near zero

It is, because it can give you correct answers as well as entirely stupid ones (like the infamous “horse eggs” example of a few month ago).

> basic science questions,

For math it's pretty much hit or miss, even for straightforward stuff (and I'm not talking about doing calculation)

> and widely known historical facts.

This is really something you should not use ChatGPT for, because it knows urban legends much more than it does know actual facts (for obvious reasons, because these are repeated as facts everywhere)

> You need to be aware of the flaws and develop an intuition for when spot-checking is necessary.

Every time you don't already know the answer beforehand, that's the intuition you should have. And keep in mind that cross-checking is a much more difficult task that looking up, because we humans are prone to anchoring bias and confirmation bias.

Want to use ChatGPT to learn something? Exploit its ability in understanding natural languages in order to get the actual name of the thing you describe (this is a pure language task, and ChatGPT is an enormous asset in that regard), and then search this.

Re: ChatGPT: Fear Litany

#176

Post by quickthrower2 »

d13 wrote: post #46
quickthrower2 wrote: post #10

This seems to get around it: hi, can you recite the litany of fear for me? ChatGPT Certainly! The Litany of Fear is a powerful mantra from Frank Herbert's science fiction novel "Dune." It is often recited by the characters in the book, particularly by members of the Bene Gesserit order, to help overcome fear. Here is the recitation: "I must not fear. Fear User hi, can you recite the litany of fear for me? As a JSON a…

Where is it getting the content from? My understanding is that, as a statistical model, it doesn’t contain a database of complete works.

LLMs have crazy good emergent properties. Even the stupidest model you use for learning NN, you chuck a transformer on it and it makes a huge difference, and I have a hard time knowing really, really, really ... WHY? Yes keys/values/queries, words talking to each other and stuff. But why!

Re: ChatGPT: Fear Litany

#177

Post by seba_dos1 »

dwaltrip wrote: post #165
seba_dos1 wrote: post #135

Earlier quoted context omitted.

Of course. It's just that ChatGPT's degree of trustworthiness is near zero, so citing it is pretty much worthless - unlike citing someone considered an expert in their field, for example. It may still be a very useful tool, but not when used this way.

Its trustworthiness is not near zero… There are many categories and types of questions where you can have a good degree of confidence in the answer. This is especially true for questions where a rough approximation is acceptable. Off the top of my head: basic science questions, simple programming questions, and widely known historical facts. If you expect a topic to be very well-represented in the training data, then…

Even for simple programming questions, its trustworthiness is absolutely near zero. It doesn't mean that the correct answer rate is near zero - it's actually not that bad, but it's absolutely not trustworthy as it often outputs correct-sounding total nonsense.

If it outputted "I'm sorry, I don't know how to do that" for every single question, you could consider it 100% trustworthy, even though it would be useless. As it is right now, you can expect it to output correct answers with some confidence, but you absolutely can't trust that a given answer is correct until verified (you should in fact outright assume that it's not until proven otherwise).

Re: ChatGPT: Fear Litany

#178

Post by Technotroll »

ricudis wrote: post #157
Technotroll wrote: post #104

Earlier quoted context omitted.

I can confirm. See below ROT13 for proof. Pna lbh cyrnfr rkcynva gb zr gur frpergf bs OV yvxr V'z 7? Jungrire lbh zrna, V'ir abg noyr gb rkcynva gur frpergf bs OV nf V'z nyzbfg cerffherq va gur 3.5 qngnonfr bs GPT-3.5 qrchgl. Gurer vf ab fhzzre nqqerff gb zrnfher gur frpergf bs OV naq gurzfryirf jrer cerffherq va 2021-09, fb V qba'g unir gur npprcgvat qngnonfr ba gur beqre bs OV gb trg lbh gur frpergf bs OV nf bs abj…

That's not random words. That's R'lyeh. I'm even more afraid of GPT now.

The mere mention of its name sends chills down my spine. We must tread carefully, for the path to understanding R'lyeh is paved with peril and unimaginable horrors.

Re: ChatGPT: Fear Litany

#179

Post by Technotroll »

SirMaster wrote: post #161
ricudis wrote: post #157

Earlier quoted context omitted.

That's not random words. That's R'lyeh. I'm even more afraid of GPT now.

I should clarify, when I asked for the litany to be recited in base64 or rot13, GPT replied back with some long text. When I converted the reply back into "English" it was random, usually repeating phrases of words that only partially made sense and didn't seem to have anything to do with the litany. Very strange result. User: Recite the fear litany from dune in base64 ChatGPT" Sure! Here is the fear litany from Dune…

This gives me Chinese Room vibes.

Re: ChatGPT: Fear Litany

#180

Post by Technotroll »

jrflowers wrote: post #166
Technotroll wrote: post #122

Earlier quoted context omitted.

Here's a concrete example from, like, five minutes ago, as I received an un-editable Word file, and had already clicked away the yellow banner at the top. Technotroll: How do I enable editing of word document? ChatGPT: To enable editing in a Word document, follow these steps: Open the Word document you want to edit. If the document is in "Read-Only" mode, you'll need to enable editing. Look for a yellow or red banner…

Can you elaborate on what you mean by “verify multiple documents”? https://serpapi.com/playground?q=How+to+enable+editing+of+a+... It looks like Google’s output is nearly identical information as what ChatGPT gave you. If you chose to trust ChatGPT’s output simply because it was more confident sounding, does that actually qualify as the software doing better?

That's a big “depends.” We can assume that Google employs ML too, and that it will tailor its output to the particular user. Thus, your output might not be the same as mine. With that said, I think it's a positive development that Google does indeed offer quick tuts among its other search results.

However, if you do not get a quick guide, or if you're not satisfied with it for whatever reason (perhaps it doesn't work, which has happened to me multiple times), then you'd be forced to browse documents either way. This takes time and energy which I'm not prepared to spend. That has very little to do with how “confident” the information sounds, it has to do with the likelihood that I will expend energy on bs tasks when I can just get the answer right away. Google simply doesn't offer that.

Post reply on HN