Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

261–270 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#261

Post by zdimension »

mock-possum wrote: post #259
__MatrixMan__ wrote: post #66

Earlier quoted context omitted.

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

“We don’t understand why neural networks work” is a myth. It’s not a miracle, it’s just code, and you can step through it to debug it the same way you would any other program.

You can step-debug a Python program using Windbg but it won't tell you a lot about what's happening since every line is hundreds of CPython API calls.

Sure, you can "step through" a neural network but all you'll ever see are arrays of meaningless floats moving around.

Re: Unexpected responses from ChatGPT: Incident Report

#262

Post by FridgeSeal »

ben_w wrote: post #215
FridgeSeal wrote: post #98

Earlier quoted context omitted.

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

"More technically, inference kernels produced incorrect results when used in certain GPU configurations." As someone who learned to read in part with the Commodore 64 user manual telling me about PEEK and POKE while I was actually in primary school , I think you're greatly overstating what primary school children write about in their holidays. Snark aside, is their message vague? Sure. But more words wouldn't actuall…

Those at least link back to a CVE, which often does have all the gory technical details.

I think your counter-example swings too far in the other direction. Nobody expects a git-diff of the fix, but a solid explanation of the whys and wherefore’s isn’t unreasonable. Cloudflare does, fly.io does, etc etc.

Re: Unexpected responses from ChatGPT: Incident Report

#263

Post by FridgeSeal »

thethimble wrote: post #152
FridgeSeal wrote: post #98

Earlier quoted context omitted.

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

If that’s what it was, they’ve done a fabulously bad job of conveying that, and then made no attempt to dig into why _that_ happened. Which, again, is like going “well it broke because it broke”, not even so much as a “this can happen because some bit of hardware had a cosmic bit flip and freaked out, generally happens with a probability of 1:x”.

Re: Unexpected responses from ChatGPT: Incident Report

#264

Post by FridgeSeal »

throwuwu wrote: post #212
thethimble wrote: post #152

Earlier quoted context omitted.

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token in…

> so my guess is the wrong precision was used in some number of GPUs

Well we wouldn’t have to guess if their PM wasn’t so pointlessly vague would we?

Re: Unexpected responses from ChatGPT: Incident Report

#265

Post by EchoChamberMan »

astrange wrote: post #260
EchoChamberMan wrote: post #239

Earlier quoted context omitted.

There are historical cases of just such a thing happening. Google warns you not to put important or private information in.

That's because they have human evaluators read user prompts to decide where to make quality improvements.

I'm pretty sure it was because people were seeing other people's data. Feel free to search for sources of you're curious.

Re: Unexpected responses from ChatGPT: Incident Report

#266

Post by jstarfish »

lynx23 wrote: post #217
FridgeSeal wrote: post #98

Earlier quoted context omitted.

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

Well, the YouTube app ChangeLog on iOS uses the same template since years: "fixed space-time continuum". This is a trend, to pretend that users are too dumb to understand the complexity of things, so just do some handwaving and thats it.

Those splines don't reticulate themselves, you know.

Re: Unexpected responses from ChatGPT: Incident Report

#267

Post by jstarfish »

renonce wrote: post #191
iforgotpassword wrote: post #161

Earlier quoted context omitted.

To be honest > On February 20, 2024, an optimization to the user experience At that point, about 10 words in, I already wanted to stop reading because it starts with the "we only wanted the best for our customers" bullshit newspeak. Anyone else going off on that stuff too? I'm pretty much already conditioned to expect whatever company is messaging me that way to take away some feature, increase pricing, or otherwise…

That sounds like a good use case for GPT. A GPT that automatically highlights such corporate speak and hints “WARNING: bullshit ahead”. I’m 100% sure it’s technically very easy to engineer such a model. Do you think OpenAI’s superalignment will ever allow you to make such a model?

You don't need a whole fucking model for this, you just need to invoke the right arcana.

Try:

> Act as a professional skeptic. Assess the passage for any deceptive, manipulative, or dishonest elements, including instances of doublespeak, inconsistency, fraud, disingenuity, deception, or sophistry:

It works on ChatGPT 3.5 as long as you're not trying to ask it questions about the Talmud. That will quickly devolve into handwavy "well, it's historically complex and difficult to understand" bullshit.

Re: Unexpected responses from ChatGPT: Incident Report

#268

Post by famouswaffles »

ggambetta wrote: post #227
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a m…

>It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion.

What would know and understand anything with this vague undefinable metric ? You ? How do i know you know and understand anything ? Can you prove that to me better than GPT can ?

Re: Unexpected responses from ChatGPT: Incident Report

#269

Post by EchoChamberMan »

astrange wrote: post #260
EchoChamberMan wrote: post #239

Earlier quoted context omitted.

There are historical cases of just such a thing happening. Google warns you not to put important or private information in.

That's because they have human evaluators read user prompts to decide where to make quality improvements.

https://arstechnica.com/security/2024/01/ars-reader-reports-...

Re: Unexpected responses from ChatGPT: Incident Report

#270

Post by polynomial »

Something about the language gives me a queasy feeling, not fully defined but feels like it has to do with how splainy [sic] the tone is, with the undertone being overpresumptuousness, or maybe it's just the frequentist in me reacting to their priors.

Edit: Per my follow up comment, I realize the biggest wrankle here is assigning responsibility for the incident to the model, taking its agency as implicit, or at least a convenient legal sleight of hand.

Post reply on HN