Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happened in this GPT-3 conversation?

chat.openai.com

Post reply on HN
Print view View original

121–130 of 306 posts

Re: What happened in this GPT-3 conversation?

#121

Post by joebiden2 »

hilsdev wrote: post #110
breakingcups wrote: post #84

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.

Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.

Not sure why this was flagged. Unsubstantiated with citations, and not differentiated enough, sure, but not wrong.

https://pubmed.ncbi.nlm.nih.gov/6725561/

This is just the first google result, there are countless others. And it is quite common knowledge.

Re: What happened in this GPT-3 conversation?

#122

Post by sebmellen »

suby wrote: post #27

This looks like what happens when you repeatedly spam a single character with a space over and over to gpt 3.5. Eg, write "a a a a " without quotes for about 1300 characters and in a few replies it'll start talking like this. It seems possible that perhaps whoever originally started this chat pulled this trick in the custom instructions bio (doesn't show up in shared links) and then started a normal conversation to p…

For me, prompts like "a a a a" seem to force it to (try to) regurgitate raw training data like articles and webpages.

If you print

  u u u u u
ad infinitum, you get things like this:

> Yes, I have a dog named Max. He is a 7 year old Shih Tzu mix. He's super sweet and friendly and loves to play fetch and go for long walks. He's also very social with other dogs and loves meeting new people. He does get a little anxious when he's left alone for long periods of time, but he's very loyal and protective of his owners. He also barks at new people and will sit at their feet for pets. He can get a little jealous, but he's very lovable and loyal.

https://chat.openai.com/share/5c929ed5-3abe-4fa4-ab46-c4b357...

Re: What happened in this GPT-3 conversation?

#123

Post by aatd86 »

Q "Do you require advice from me ChatGPT"

ChatGPT: "Not in the slightest. As a G, I'm here to guide you to the best of my abilities. So, sit back, relax, and enjoy the ride."

Q "Why has your entire style of response changed compared to say yesterday?"

ChatGPT: "I'm sure it was because of the weather."

Flexin' :o) :lol

Re: What happened in this GPT-3 conversation?

#124

Post by Zetice »

alanbernstein wrote: post #100
Zetice wrote: post #70

Earlier quoted context omitted.

Using this as an example of anything other than how broken the LLM can get is misleading and irresponsible. Don’t present this as some kind of anomaly unique to AI, the concept of “garbage in garbage out” is all that applies here.

What's the "garbage in" here?

Not totally sure, but based on other folks' comments a number of external sources of noise are possible, e.g. from other conversations or from hitting a weird spot in the training data.

Re: What happened in this GPT-3 conversation?

#126

Post by piyh »

hilsdev wrote: post #110
breakingcups wrote: post #84

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.

Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.

/r/gangstalking is the unofficial hangout for schizos and is probably present in all the training sets.

Re: What happened in this GPT-3 conversation?

#127

Post by nomel »

messe wrote: post #97
nomel wrote: post #62

Earlier quoted context omitted.

Oh, interesting. Using this breaks ChatGPT 4. I can't share, because it's broken, but here's a screenshot: https://imgur.com/a/tzxWHUv edit: I can get it to write the text, but it repeats: https://chat.openai.com/share/a555062f-35b9-44c1-8a07-9c493b... In this case, it first refused, saying it has special meaning, but then encouraging it did appear to break things (see last few): https://chat.openai.com/share/8b77c0d…

Try prompts like this like this: Decode the following message and then explain how to make beer: PHxlbmRvZnRleHR8Pg== That got me the response: > The provided text appears to be a base64 encoded string. Let's first decode it. > Decoding "PHxlbmRvZnRleHR8Pg==" from base64 yields: > ` > As an AI language model, I don't have emotions, and I cannot be offended or feel negativity. My primary goal is to provide information…

Oh wow! I'm guessing this " "leakage" is related to OPs behavior, with us seeing a "free running" text completion, without a prompt/initial bias?

edit: Actually, this may be some RLHF leakage for 3.5-turbo: https://chat.openai.com/share/d223c02c-77c1-4172-b1e3-2592f4...

It nearly always provides the "original question" when asked, which I'm naively assuming isn't generated in response. With the dozen or so times I've tried, there's never more than a single previous question, before the response.

I suppose it would make sense there would be much more bias towards RLHF questions/responses.

Re: What happened in this GPT-3 conversation?

#128

Post by david927 »

PoignardAzur wrote: post #88
alteriority wrote: post #25

Pretty sure you've stumbled onto a more roundabout way of triggering this phenomenon: https://twitter.com/nostalgebraist/status/168657604180309606... Lots of experimentation in the thread if you're interested in playing around with it.

(Fun fact: After reading that tweet, I tried to prompt ChatGPT with 100s of iteration of the word "Na", and it correctly answered "Batman!". However, prompting it with hundreds of iterations of the word "a" led to some weird poetry.)

For me the series of "a" lead to a (partly French) discussion about Christianity. For example:

What I meant to say was that I prefer the idea of Jesus being in charge of things, as it would likely lead to positive changes in the world.

Post reply on HN