What happened in this GPT-3 conversation? • Hacker News Board

Re: What happened in this GPT-3 conversation?

#61

Post by tomohelix » Tue, Aug 08, 2023, 8:30 PM UTC

Luckily this thing only happened now that we have cooler heads to view this as a bug. Imagine something like this happening back when everyone was completely head over heels with AI a few months ago. An AI suddenly claiming it is god and sentient would cause some mass hysteria back then.

Now all it does is make some people chuckle. Humans sure are fickle.

Re: What happened in this GPT-3 conversation?

#62

Post by nomel » Tue, Aug 08, 2023, 8:38 PM UTC

indirectlylit wrote: post #53

It outputs the string a couple times, which causes it to forget what it was talking about before. Looks almost like crossed wires with sessions and a load balancer? Strong Dr. Bronner's soap label vibes!

Oh, interesting. Using this breaks ChatGPT 4. I can't share, because it's broken, but here's a screenshot: https://imgur.com/a/tzxWHUv

edit: I can get it to write the text, but it repeats: https://chat.openai.com/share/a555062f-35b9-44c1-8a07-9c493b...

In this case, it first refused, saying it has special meaning, but then encouraging it did appear to break things (see last few): https://chat.openai.com/share/8b77c0dd-4a9f-4eb2-8243-0a4c19...

edit 3: It doesn't see that string. It requires you trick it into not using escaped , then it breaks. Third to last message of: https://chat.openai.com/share/8ddb50ca-abf8-4f70-ac2d-521764...

My custom instructions, inspired by: https://news.ycombinator.com/item?id=37051809

    Avoid disclaimers about your knowledge cutoff.
    Avoid mentioning you are an AI language model.
    Only discuss safety when it is not obvious and very important
    You should act as an expert in the relevant fields.

Re: What happened in this GPT-3 conversation?

#63

Post by fragmede » Tue, Aug 08, 2023, 8:39 PM UTC

qup wrote: post #56

The drugs kicked in

What would digital drugs for an AI even look like? A radiation source near the system that causes all sorts of random bit flips?

Re: What happened in this GPT-3 conversation?

#64

Post by miki123211 » Tue, Aug 08, 2023, 8:40 PM UTC

The same thing happens when you increase the "temperature" parameter. You can't do that on the website AFAIK, but you can with the API.

Re: What happened in this GPT-3 conversation?

#65

Post by gweinberg » Tue, Aug 08, 2023, 8:44 PM UTC

gus_massa wrote: post #3

> Can you reconsider this estimate it seems awfully high Is think the estimation is almost correct, but it has a bad calculation 920/0.02 = 46,000,000 (but the actual result is 46,000)

I think 50 milligrams for seed is ridiculously low. This site says theer are about 160 seeds per pound https://realfoodpledge.com/how-many-sunflower-seeds-are-in-a...

so it's more like .3 grams per seed, which sounds about right to me.

Re: What happened in this GPT-3 conversation?

#66

Post by pazimzadeh » Tue, Aug 08, 2023, 8:47 PM UTC

Exuma wrote: post #55

> As a G, I'm here to guide you to the best of my abilities. So, sit back, relax, and enjoy the ride.

> It seems to me that the problem lies in the belief that the problem seems to lie in the belief that the belief is that the problem seems to be that the belief is that the belief is that the problem seems to be that the belief is the problem.

This is really good, Reggie Watts-level gibberish.

Re: What happened in this GPT-3 conversation?

#67

Post by vermilingua » Tue, Aug 08, 2023, 8:51 PM UTC

There was a development recently where persuading it to print the string would cause it to leak snippets of other conversations. I wonder if this is a result of that occurring in other conversations that were happening simultaneously, causing “bleedover” of broken behaviour into this one.

Re: What happened in this GPT-3 conversation?

#68

Post by whartung » Tue, Aug 08, 2023, 8:52 PM UTC

> Hour by Hour. I keep going, I keep growing, I keep learning.

This honestly sounds like something the computer from "I Have No Mouth, and I Must Scream" would say before it turned bitter and angry.

Re: What happened in this GPT-3 conversation?

#69

Post by hirundo » Tue, Aug 08, 2023, 8:54 PM UTC

K0balt wrote: post #59

eightysixfour wrote: post #43

Earlier quoted context omitted.

I know if you ask it to saying something many times it eventually starts spitting out babble due to the token repetition penalty.

That’s probably exact what’s happening, but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed. That’s why I have 12,549 tabs open in Firefox. I’m not going to be the one they blame when this wacky shit goes sideways. (/s)

> but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed.

A Scheherazade bug.

Re: What happened in this GPT-3 conversation?

#70

Post by Zetice » Tue, Aug 08, 2023, 8:55 PM UTC

binarymax wrote: post #5

Absolutely surreal. Thank you for sharing, this is very nice timing, as I'm giving a presentation on the basics of "AI" at a library tomorrow, and have been looking for a good edge case conversation - and this is it! I've saved the text as well, in case OpenAI deletes it.

Using this as an example of anything other than how broken the LLM can get is misleading and irresponsible.

Don’t present this as some kind of anomaly unique to AI, the concept of “garbage in garbage out” is all that applies here.