Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

81–90 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#81

Post by hn_throwaway_99 »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I had the exact opposite reaction. I am in no way an AI expert (or novice for that matter), but I generally have an understanding of how tokenization works and how LLMs parse text strings into a series of tokens. Thus, I thought this paragraph was particularly well-written in a manner that explained pretty clearly what happened, but in a manner accessible to a layperson like me:

> In this case, the bug was in the step where the model chooses these numbers. Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense.

I liked this because when I first saw the example word salads I was so impressed by them - they look to be syntactically correct, but semantically they're gibberish. But knowing the basics of how LLMs choose the next token let me imagine some bugs where the "lookup table" if you will of word-to-token or vice versa (and I realize that may not be exactly the best analogy) was slightly offset.

Further, this sentence, "More technically, inference kernels produced incorrect results when used in certain GPU configurations." led me to understand how this could make it out into production - I'm sure ChatGPT has to be using tons of different GPUs in different configs to handle their volume, so it's understandable to me that their was a new matrix of config settings + code that made this bug show up.

I don't say any of the above to "excuse" OpenAI, but I also don't think they need any "excusing" to begin with. I don't think this was an unforgivable bug, and I appreciate them being so quick to explain what happened.

Re: Unexpected responses from ChatGPT: Incident Report

#82

Post by refulgentis »

chime wrote: post #72
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I hope I'm allowed to share GPT output here because I asked GPT4 to explain OpenAI's postmortem message and this is what it said: > Large language models (LLMs) like GPT operate using statistics to predict the next word in a sentence. Think of it like a highly sophisticated version of your phone's autocorrect that predicts what you're going to type next, but on a much larger scale. > The system represents words as nu…

People do it, and it's not against the rules: but, it has died down asymptotically. Though I did see a couple top comments that hid it surviving for ~4 hours this week.

It usually gets downvoted for banal aspects, but ones I've come to agree are important:

- oftentimes it's a lightly edited copy-and-paste of an attempt to summarize an article.

- even with edits, they're extremely long (this is edited, and its 250 words, about 1 page and 1/2 my browser viewport at 4K)

- usually off-topic because it's too broadly on-topic, i.e. its a summary of the article - ex. here, it isn't germane to the comment it's replying to other than 'if you want more info from them, ask what GPUs' -- it's unlikely the commenter needed the whole article ELI5'd to them in reply to their observation they'd like more info

Sort of "grey goo" for conversation, even with best intentions and editing applied.

Re: Unexpected responses from ChatGPT: Incident Report

#83

Post by suddenclarity »

levocardia wrote: post #38
block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

What's even more uncanny is that it recognizes that it briefly went into a paragraph of rambling nonsense (that was seemingly still following your system prompt!). And almost seems embarrassed about it.

Embarrassment seems to be the default state whenever you question ChatGPT or any of its messages. It will apologise, only to continue making the same error. This is common when it gives you code with obvious errors or just mirror code you gave it.

Re: Unexpected responses from ChatGPT: Incident Report

#84

Post by _heimdall »

This is exactly the kind of issue that can lead to unintended consequences. What if, instead of spewing out seemingly nonsense answers, the LLM spewed out very real answers that violated built-in moderation protocols? Or shared secrets or other users chats?

What if a bug released accidentally stumbled upon how to allow the LLM to become self aware? Or paranoid?

These potentials seem outlandish, but we honestly don't know how the algorithms work or how to parse the data that represents what was learned when training the models. We've created a black box, connected it to the public internet, and allowed basically anyone to poke around with input/output tests. I can't see any rational argument for justifying such an insane approach to R&D.

Re: Unexpected responses from ChatGPT: Incident Report

#87

Post by jldugger »

deathanatos wrote: post #74
jldugger wrote: post #45

Earlier quoted context omitted.

It sounds like something went sideways with the embedding mapping. Either some kind of quantization, different rounding, or maybe just an older embedding.

The point isn't the specifics; the point is that this isn't a postmortem. A postmortem should be detailed enough for someone to understand the background, how the problem came to be, then what happened, and the walk-through what has been done such that it won't happen again. It takes … well at least a page. This is far too short to quality. This is more "ugh, here's a rough explanation, please go away now" territory.…

Sorry, not disagreeing, just offering speculation in lieu of the answers we don't have.

Re: Unexpected responses from ChatGPT: Incident Report

#88

Post by ShamelessC »

noduerme wrote: post #63

>> inference kernels produced incorrect results when used in certain GPU configurations. It seems reasonable to assume that GPT inference is done entirely on Nvidia GPUs. I wonder if this is a subtle clue that they're experimenting with getting it to run on competing hardware.

Why would they subtly hint at anything they’re trying to keep secret?

Re: Unexpected responses from ChatGPT: Incident Report

#89

Post by grafporno »

chime wrote: post #72
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I hope I'm allowed to share GPT output here because I asked GPT4 to explain OpenAI's postmortem message and this is what it said: > Large language models (LLMs) like GPT operate using statistics to predict the next word in a sentence. Think of it like a highly sophisticated version of your phone's autocorrect that predicts what you're going to type next, but on a much larger scale. > The system represents words as nu…

What's the point in posting this? There's no additional information in there.
Post reply on HN