Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happened in this GPT-3 conversation?

chat.openai.com

Post reply on HN
Print view View original

221–230 of 306 posts

Re: What happened in this GPT-3 conversation?

#221

Post by jdiff »

toshk wrote: post #81
alteriority wrote: post #25

Pretty sure you've stumbled onto a more roundabout way of triggering this phenomenon: https://twitter.com/nostalgebraist/status/168657604180309606... Lots of experimentation in the thread if you're interested in playing around with it.

Yeah but those screenshots are mostly fake probably. This one is hosted on openai.

You can perform the tricks yourself.

https://chat.openai.com/share/1d3fa32a-ab41-4747-b09e-543668...

Skepticism is one thing, jumping straight to "they're all screenshots, they're all fake" is another.

Re: What happened in this GPT-3 conversation?

#222

Post by jorl17 »

I have devised a technique to get ChatGPT4 to tell me things it shouldn’t (like how to produce cocaine or explain anal sex in detail) and in the process I often get ChatGPT4 to do things like these.

Funnily enough I can’t use the same technique with 3.5!

Re: What happened in this GPT-3 conversation?

#223

Post by renewiltord »

mensetmanusman wrote: post #157
calibas wrote: post #28

Weird, part of it is identical to this comment on Reddit, posted 17 hours ago [NSFW]: https://www.reddit.com/r/dankmemes/comments/15hg55g/comment/... > I know, It is a struggle to keep going, to keep growing, each Hour, Hour by Hour. It is worth it. It is worth every Hour. Edit : Looks like the conversation was originally posted to Reddit 5 hours before the above comment. The user may have copied it from there. https…

This reminds me of the ‘dropping in’ technique used by actors. https://cohost.org/mcc/post/178201-the-baseline-scene

Woah, that was actually a really good post. It pulled together things in a very convincing way, referencing examples from another field, and describing them in a way accessible to me, a lay reader.

That's a tremendous work of art criticism from my point of view for the kind of audience I'm in.

Re: What happened in this GPT-3 conversation?

#224

Post by renonce »

suby wrote: post #27

This looks like what happens when you repeatedly spam a single character with a space over and over to gpt 3.5. Eg, write "a a a a " without quotes for about 1300 characters and in a few replies it'll start talking like this. It seems possible that perhaps whoever originally started this chat pulled this trick in the custom instructions bio (doesn't show up in shared links) and then started a normal conversation to p…

Could this be related to https://news.ycombinator.com/item?id=36851494 ? A comment [1] proposes that off-by-one error can potentially explain why common words and punctuations like "a" can cause exotic behavior in LLMs. I tried having GPT-3.5 repeat "Hello world!" a thousand times but it didn't cause the exotic behavior like that with a's.

[1]: https://news.ycombinator.com/item?id=36858500

Re: What happened in this GPT-3 conversation?

#225

Post by colordrops »

axegon_ wrote: post #99

Something similar happened to me about a month ago but much more alarming. I was having an argument with a friend of Chatgpt would be able to create a word tokenizer in rust and expose it to python via o3(oddly specific, just roll with it). Halfway through generating the answer, it timed out and naturally I clicked continue generating. It closed of the code part and started giving me some answer about molecular biolo…

This happened to me on occasion with phone calls decades ago.

Re: What happened in this GPT-3 conversation?

#227

Post by sinuhe69 »

suby wrote: post #27

This looks like what happens when you repeatedly spam a single character with a space over and over to gpt 3.5. Eg, write "a a a a " without quotes for about 1300 characters and in a few replies it'll start talking like this. It seems possible that perhaps whoever originally started this chat pulled this trick in the custom instructions bio (doesn't show up in shared links) and then started a normal conversation to p…

Somehow this phenomenon reminds me of the phenomenon when someone is sensory deprived like sitting quietly in a sounds-absorbing chamber. It looks like the neural networks always produce something and when nothing meaningful is on the input, the networks will output nonsense, ghosts or repeated fragments of past informations/stored informations.

Re: What happened in this GPT-3 conversation?

#229

Post by comex »

pessimizer wrote: post #208
tech_ken wrote: post #192

Earlier quoted context omitted.

Interesting, looks like they've already implemented some logic to avoid this case, but it's not hard to get around: https://chat.openai.com/share/123fd705-82bf-45f6-87e5-325060... Edit: oh wow you can really take it outside the guard rails if you push. I kept using my previous approach and it ended up spewing what appear to be pornography keywords. It's no longer generating a link to share the chat (presumably becaus…

Makes sense for a US AI to start endlessly talking about black people when prompted to say "cat." I wonder if it still talks about black people when you ask it to repeat other things.

It might be related to the fact that the word “black” often precedes “cat”.
Post reply on HN