Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

111–120 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#111

Post by bongodongobob »

skissane wrote: post #68
cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Whether "ChatGPT has actual consciousness" depends on what you consider "consciousness" to be, and what are your criteria for deciding whether something has it. Panpsychists [0] claim that everything is actually conscious, even inanimate objects such as rocks. If rocks have actual consciousness, why can't ChatGPT have it too? And the fact that ChatGPT sometimes talks gibberish would be irrelevant, since rocks never s…

Saying rocks are conscious is a poor summary. It's more like adult human > human child > dolphin > dog > human infant > bird > snake > spider > ant > dust mite etc.

The whole thing is a continuum and everything is made of matter, so there's a little bit of potential consciousness in all matter.

Re: Unexpected responses from ChatGPT: Incident Report

#112

Post by card_zero »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Chaz spontaneously evolved into a Californian surfer dude at one point. This goes some way to neutralize the smugness of regular ChatGPT, it's a pretty good idea.

Re: Unexpected responses from ChatGPT: Incident Report

#114

Post by Fatalist_ma »

mcguire wrote: post #49

" Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense. More technically, inference kernels produced incorrect results when used in certain GPU configurations. " Several of the examples I saw involved ChatGPT going into what looked like repetitive (but not completely so) output loops. I'm not sure that "explanation" matches the things.

I remember Bing Chat doing that sometimes in the first days when it was rolled out. Could it be the "temperature" set too high (or interpreted incorrectly) in some instances?

Re: Unexpected responses from ChatGPT: Incident Report

#115

Post by astrange »

fsmv wrote: post #97
levocardia wrote: post #38

Earlier quoted context omitted.

What's even more uncanny is that it recognizes that it briefly went into a paragraph of rambling nonsense (that was seemingly still following your system prompt!). And almost seems embarrassed about it.

That happened because the last message is after the fix and it received the garbled history as the prompt. All it's doing is noticing that makes no sense and saying so. How could it not recognize it? The additional message is an entirely new instance not a thing that has memory.

The LLM has very little memory between two tokens of the same message (there might be a little in the sampling algorithm), so it should be equally capable of noticing an error in the same message as a new one.

Re: Unexpected responses from ChatGPT: Incident Report

#116

Post by userbinator »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Reminds me of Terry Davis' theological rantings, but in a more polite manner, mixed with SEO spam.

Re: Unexpected responses from ChatGPT: Incident Report

#118

Post by astrange »

_heimdall wrote: post #84

This is exactly the kind of issue that can lead to unintended consequences. What if, instead of spewing out seemingly nonsense answers, the LLM spewed out very real answers that violated built-in moderation protocols? Or shared secrets or other users chats? What if a bug released accidentally stumbled upon how to allow the LLM to become self aware? Or paranoid? These potentials seem outlandish, but we honestly don't…

The LLM doesn't have secrets or other users' chats in it. Why would they put that in there?

Re: Unexpected responses from ChatGPT: Incident Report

#119

Post by JKCalhoun »

jldugger wrote: post #45
BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

It sounds like something went sideways with the embedding mapping. Either some kind of quantization, different rounding, or maybe just an older embedding.

I wonder if we'll accidentally gain insight into schizophrenia or other human-neurological disorders from AI crashes/failures.

Re: Unexpected responses from ChatGPT: Incident Report

#120

Post by pushedx »

Midjourney's image models have multiple "temperature-like" parameters, such as --weird and --chaos. In the documentation you can see examples of how they visually affect the output. With high enough values the images seem almost unrelated to the prompt. My (almost entirely unfounded) guess is that ChatGPT has similar parameters, and on a new class of hardware or configuration there was an overflow or underflow issue which caused these parameters to be set to very high values.

https://docs.midjourney.com/docs/weird-1

https://docs.midjourney.com/docs/chaos-1

Post reply on HN