Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

121–130 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#121

Post by ummonk »

hn_throwaway_99 wrote: post #81
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I had the exact opposite reaction. I am in no way an AI expert (or novice for that matter), but I generally have an understanding of how tokenization works and how LLMs parse text strings into a series of tokens. Thus, I thought this paragraph was particularly well-written in a manner that explained pretty clearly what happened, but in a manner accessible to a layperson like me: > In this case, the bug was in the ste…

No, that just explains the symptom of the bug, not the underlying bug, how it came about, and how they can prevent it from happening again.

"More technically, inference kernels produced incorrect results when used in certain GPU configurations" has zero technical detail. The only information it is providing us is that the bug only showed up in some GPU configurations.

Re: Unexpected responses from ChatGPT: Incident Report

#122

Post by someotherperson »

transcriptase wrote: post #41
okdood64 wrote: post #10

Say what you will about Google, but bugs such as these, released to the public (or their enterprise customers) this debilitating to a core product, are exceedingly rare.

I can tell you don’t have Google home. Responding with complete nonsense is now the status quo for that product. Myself and everyone I know have gone from complex routines, smart home control, and interacting with calendars… to basically using it as a voice activated radio since it can’t be trusted for much else. Timers? Might get set, might not. Might tell you it’s set then cancel itself. Might activate in a differe…

Is that how bad it got? I was using one up to around maybe 2019-2020 and it seemed pretty good at the time, definitely didn’t experience a lot of what you did

Re: Unexpected responses from ChatGPT: Incident Report

#123

Post by reaperducer »

dimatura wrote: post #64
BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

Yeah, definitely opaque. If I had to guess it sort of sounds like a code optimization that resulted in a numerical error, but only in some GPUs or CUDA versions. I've seen that sort of issue happen a few times in the pytorch framework, for example.

Yeah, definitely opaque.

I wonder what the AI would say if someone asked it what happened.

It would be pretty funny if it gave a detailed answer.

Re: Unexpected responses from ChatGPT: Incident Report

#124

Post by ummonk »

cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Humans do the same thing when they get a stroke. Does that mean they don't have actual consciousness?

Re: Unexpected responses from ChatGPT: Incident Report

#125

Post by peddling-brink »

reaperducer wrote: post #123
dimatura wrote: post #64

Earlier quoted context omitted.

Yeah, definitely opaque. If I had to guess it sort of sounds like a code optimization that resulted in a numerical error, but only in some GPUs or CUDA versions. I've seen that sort of issue happen a few times in the pytorch framework, for example.

Yeah, definitely opaque. I wonder what the AI would say if someone asked it what happened. It would be pretty funny if it gave a detailed answer.

It will make something up if it answers at all. It doesn’t know.

Re: Unexpected responses from ChatGPT: Incident Report

#126

Post by ummonk »

astrange wrote: post #118
_heimdall wrote: post #84

This is exactly the kind of issue that can lead to unintended consequences. What if, instead of spewing out seemingly nonsense answers, the LLM spewed out very real answers that violated built-in moderation protocols? Or shared secrets or other users chats? What if a bug released accidentally stumbled upon how to allow the LLM to become self aware? Or paranoid? These potentials seem outlandish, but we honestly don't…

The LLM doesn't have secrets or other users' chats in it. Why would they put that in there?

If they use batching during inference (which they very probably do), then some kind of coding mistake of the sort that happened with this bug absolutely could result in leakage between chats.

Re: Unexpected responses from ChatGPT: Incident Report

#127

Post by card_zero »

bongodongobob wrote: post #111
skissane wrote: post #68

Earlier quoted context omitted.

Whether "ChatGPT has actual consciousness" depends on what you consider "consciousness" to be, and what are your criteria for deciding whether something has it. Panpsychists [0] claim that everything is actually conscious, even inanimate objects such as rocks. If rocks have actual consciousness, why can't ChatGPT have it too? And the fact that ChatGPT sometimes talks gibberish would be irrelevant, since rocks never s…

Saying rocks are conscious is a poor summary. It's more like adult human > human child > dolphin > dog > human infant > bird > snake > spider > ant > dust mite etc. The whole thing is a continuum and everything is made of matter, so there's a little bit of potential consciousness in all matter.

I'm trying to think of a civil and constructive way to say "bullshit".

I guess an obvious objection is "why?". Then something about Russell's teapot. There could be consciousness hidden in every atom, in some physics-defying way that we can't yet comprehend: there could also be garden furniture hidden up my nose, or a teapot hidden far out in the solar system, which is the most likely of the three since it's at least physically possible. Why think any of these things?

https://en.wikipedia.org/wiki/Russell%27s_teapot

Re: Unexpected responses from ChatGPT: Incident Report

#128

Post by transcriptase »

someotherperson wrote: post #122
transcriptase wrote: post #41

Earlier quoted context omitted.

I can tell you don’t have Google home. Responding with complete nonsense is now the status quo for that product. Myself and everyone I know have gone from complex routines, smart home control, and interacting with calendars… to basically using it as a voice activated radio since it can’t be trusted for much else. Timers? Might get set, might not. Might tell you it’s set then cancel itself. Might activate in a differe…

Is that how bad it got? I was using one up to around maybe 2019-2020 and it seemed pretty good at the time, definitely didn’t experience a lot of what you did

It was fantastic 2017-2020. At one point I had 8 or 9 around the house and loved it. But one by one, every single feature or voice command I would use would either stop working or become unpredictable. Integrations with other hardware or companies would cease without warning. Latency became more pronounced.

And of course being Google there’s no such thing as a changelog, so everyone is left guessing what they’ve changed and whether there’s a passable workaround.

r/googlehome is a sight to behold when viewed as a testament to how to slowly ruin a product.

Re: Unexpected responses from ChatGPT: Incident Report

#129

Post by throwaway920102 »

Some samples I found quickly scanning the ChatGPT subreddit for anyone curious:

https://www.reddit.com/r/ChatGPT/comments/1awm8kv/chatgpt_sp...

^ quick non sequiturs

https://www.reddit.com/r/ChatGPT/comments/1aw1o9x/i_had_a_re...

https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....

^ incredibly trippy text that devolves into strangely formatted comma separated text that almost resembles some code / amateur cryptography with multiple languages/alphabets

https://www.reddit.com/r/ChatGPT/comments/1avydjd/anyone_els...

^ multiple responses in the same conversation that start off fairly normal but end up unhinged each time

https://www.reddit.com/r/ChatGPT/comments/1avyr09/first_time...

^ business/tech jargon nonsense soup

https://www.reddit.com/r/ChatGPT/comments/1awtvq6/descent_in...

^ purple prose nonsense

https://www.reddit.com/r/ChatGPT/comments/1aws3j1/what_in_th...

^ techy/business/purple prose-y nonsense

It's unfortunate that Open AI seems to be deleting these from user's histories, they are really fascinating as almost a "peek under the hood" into what crappy outputs and failure modes can look like. Some of the fake words / invented language really passes as stuff that you'd believe were real English words if you googled them, but when you do they have no definition or search results, which is trippy.

The one that has the cryptographic looking / symbolic looking messaging is really spooky, I could see someone suffering from delusions/mental illness believing there was some deeper meaning to the text and really going to bad places. Honestly makes me think we might have real problems in the future with people reading into gibberish as if its mystical/prophetic. People already use Tarot or Astrology similarly and its not nearly as neat.

Post reply on HN