Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

211–220 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#211

Post by nottorp »

renonce wrote: post #191
iforgotpassword wrote: post #161

Earlier quoted context omitted.

To be honest > On February 20, 2024, an optimization to the user experience At that point, about 10 words in, I already wanted to stop reading because it starts with the "we only wanted the best for our customers" bullshit newspeak. Anyone else going off on that stuff too? I'm pretty much already conditioned to expect whatever company is messaging me that way to take away some feature, increase pricing, or otherwise…

That sounds like a good use case for GPT. A GPT that automatically highlights such corporate speak and hints “WARNING: bullshit ahead”. I’m 100% sure it’s technically very easy to engineer such a model. Do you think OpenAI’s superalignment will ever allow you to make such a model?

... but on the contrary, all the public services are programmed to bullshit you.

Every time I ask a question, the answer could be 3x smaller. Even if it's right and not a hallucination.

Re: Unexpected responses from ChatGPT: Incident Report

#212

Post by throwuwu »

thethimble wrote: post #152
FridgeSeal wrote: post #98

Earlier quoted context omitted.

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token instead. Instant aphasia. The report said it was a GPU configuration problem so my guess is the wrong precision was used in some number of GPUs but I have no idea how they configure their cluster so take that with a giant grain of salt.

Re: Unexpected responses from ChatGPT: Incident Report

#213

Post by mavamaarten »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

Honestly, I don't think we're entitled to a deep technical write-up. We are users of their product, they messed something up, and that's it. Unless they signed an SLA agreement, it's just shit that happens. They don't have to give a technical view into their internal kitchen if they don't want to.

Re: Unexpected responses from ChatGPT: Incident Report

#214

Post by cqqxo4zV46cp »

throwuwu wrote: post #212
thethimble wrote: post #152

Earlier quoted context omitted.

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token in…

Yep. Some obviously made their mind up before the page even rendered.

Re: Unexpected responses from ChatGPT: Incident Report

#215

Post by ben_w »

FridgeSeal wrote: post #98
__MatrixMan__ wrote: post #66

Earlier quoted context omitted.

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

"More technically, inference kernels produced incorrect results when used in certain GPU configurations."

As someone who learned to read in part with the Commodore 64 user manual telling me about PEEK and POKE while I was actually in primary school, I think you're greatly overstating what primary school children write about in their holidays.

Snark aside, is their message vague? Sure. But more words wouldn't actually tell us more unless they also published a lot more about their setup — "we were casting float16 to uint16 on line 249 of server_config.c" isn't going to really help either.

Also, here's a recent security update from Apple to compare against: https://support.apple.com/en-gb/HT214056

Re: Unexpected responses from ChatGPT: Incident Report

#216

Post by redskyluan »

Lessons learned:

The Importance of Model Agnosticism: With the rapid evolution of AI models, building applications that are model-agnostic has become more critical than ever.

Control and Interpretability Matter: Relying solely on large language models (LLMs) poses significant challenges for creating applications that can be deployed in real-world scenarios.

The Need for Open Models: Lastly, the push for more open models has never been more apparent. Open-source models are essential for fostering innovation, ensuring accessibility, and maintaining the integrity of our work in the AI field.

Re: Unexpected responses from ChatGPT: Incident Report

#217

Post by lynx23 »

FridgeSeal wrote: post #98
__MatrixMan__ wrote: post #66

Earlier quoted context omitted.

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

Well, the YouTube app ChangeLog on iOS uses the same template since years: "fixed space-time continuum". This is a trend, to pretend that users are too dumb to understand the complexity of things, so just do some handwaving and thats it.

Re: Unexpected responses from ChatGPT: Incident Report

#218

Post by Lerc »

MichaelDickens wrote: post #146
block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Why is the nonsense paragraph still grammatical? I thought the tokens were beings elected more or less at random.

I'm wondering if it has something to do with the number range of the values being chosen.

For example if a configuration ended up using a fp16/bfloat16 to store an index into the token table (byte pair encoded) then you would get errors only when the tokens had larger numbers.

The text

"Keep it at the dial, in right on Pitch."

comes to 19999 433 520 279 28960 11 304 1314 389 46676 13

Of which only Keep, dial, and Pitch would be large enough to be susceptible to precision errors. That would enable it to generate valid enough sentence structure.

Re: Unexpected responses from ChatGPT: Incident Report

#219

Post by nottorp »

makin wrote: post #184
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Aren't we also machines? If we make theories that discount our own intelligence that's just throwing our hands up and giving up. Maybe I would agree with you if LLMs weren't already embedded in a lot of useful products. I myself measurably save a lot of time using ChatGPT in such "unintelligent conversations". It's intelligent in the ways that matter for a tool.

IMO you're using the wrong term or have a low bar for 'intelligence'.

It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't.

That may be intelligence but it doesn't justify the religious like tone some people use when talking about LLMs.

Re: Unexpected responses from ChatGPT: Incident Report

#220

Post by rsynnott »

nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

> People are talking about ChatGPT hallucinating

It's really unfortunate that some much of the terminology around LLMs is so anthropomorphic; it's extremely misleading.

(People sometimes suggest replacing this one with confabulating, but I'm not really sure that's much better)

Post reply on HN