Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

31–40 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#31
I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis)

https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f...

EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun.

EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6PvM

Re: Unexpected responses from ChatGPT: Incident Report

#32

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

kind of amazing really, like speaking in tongues

Re: Unexpected responses from ChatGPT: Incident Report

#33
post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

I have no skin on the consciousness game but those were not "random" words and humans do something similar when they are mentally ill. https://en.wikipedia.org/wiki/Clanging

Not to mention that a hypothetical "conscious" system that works by emitting token probabilities will still sound completely random if you do not choose the tokens according to the emitted probabilities.

Re: Unexpected responses from ChatGPT: Incident Report

#34
post #9
post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

I don’t think it has conciousness, but your argument is not very strong, this is more akin to a sensory problem. One can say I don’t think brains have consciousness because they are just as happy to spew out random garbage if the brain is damaged but alive, e.g. aphasia where involuntary use of incorrect words occurs.

I think what they mean is, the model was unable to recognize that "it" was having issues, so it's not self-aware in that sense.

We can all calm down a little, it's ok.

Re: Unexpected responses from ChatGPT: Incident Report

#36
post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Posting one more time: this is proof that AI is connected to human-like linguistic patterns, IMO. No, it obviously doesn’t have “consciousness” in the sense of an ongoing stream-of-consciousness monologue, but that doesn’t mean it’s not mimicking some real part of human cognition.

https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...

Re: Unexpected responses from ChatGPT: Incident Report

#37

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

EDIT: misread

Re: Unexpected responses from ChatGPT: Incident Report

#38

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

What's even more uncanny is that it recognizes that it briefly went into a paragraph of rambling nonsense (that was seemingly still following your system prompt!). And almost seems embarrassed about it.

Re: Unexpected responses from ChatGPT: Incident Report

#39
post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

It literally doesn’t matter. If it effectively quacks like a duck then ducks are fucked. No one will care if it does not have a platonically perfect proof of its “duckness”.

“If it quacks like a duck, then ducks are fucked” is a beautiful modern twist on the phrase that improves it 1000x. Thank you - will be remembering this one as AI zooms along

Re: Unexpected responses from ChatGPT: Incident Report

#40

Say what you will about Google, but bugs such as these, released to the public (or their enterprise customers) this debilitating to a core product, are exceedingly rare.

I can think of at least four open consumer bugs in YouTube and Android Auto that have persisted for years.
Post reply on HN