Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

241–250 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#241

Post by meowface »

throwuwu wrote: post #212
thethimble wrote: post #152

Earlier quoted context omitted.

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token in…

Agreed, but I think they raise a fair point that some kind of automated testing probably should've existed to look for sudden major changes in prompt -> output production between versions, with lots of prompt test cases. Maybe this testing did exist but this problem just didn't surface, for some reason?

Re: Unexpected responses from ChatGPT: Incident Report

#242

Post by jasonjmcghee »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

Huh. Thought it was pretty clear. I read it as:

~ "We tried a new optimization technique that modifies how next token candidates are chosen and there was a bug."

That would definitely produce the behavior people saw and makes sense.

Re: Unexpected responses from ChatGPT: Incident Report

#243

Post by Const-me »

Speaking about random sampling during inference, most ML models are doing it rather inefficiently.

Here’s a better way: https://github.com/Const-me/Cgml/blob/master/Readme.md#rando...

My HLSL is easily portable to CUDA, which has `__syncthreads` and `atomicInc` intrinsics.

Re: Unexpected responses from ChatGPT: Incident Report

#244

Post by crote »

ggambetta wrote: post #227
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a m…

I believe that for a large number of applications the distinction between "knowing something" and "containing knowledge" doesn't really matter.

In 2010s Google I could type in a query, and it would redirect me to a human-written answer on something like StackOverflow with hopefully the right answer; in 2024 ChatGPT you can type in a query and it'll mash together some relevant human-written snippets into something which is hopefully coherent, relevant, and correct. Sure, it's "just" a Markov chain generator on steroids, but it's still giving you useful output.

I do agree that it is a bit worrying to what extent people are anthropomorphizing it, and treating it like something with an actual opinion or an unquestionable source-of-truth. The "I asked ChatGPT" comments should be treated the same as "I put it into Google and this was the first result": it's just line noise and doesn't add anything to the conversation. If you don't have anything to add yourself, why comment at all?

Re: Unexpected responses from ChatGPT: Incident Report

#245

Post by dns_snek »

mindcrime wrote: post #80
DonHopkins wrote: post #4

ChatTHC

You inspired me to try an experiment just now: Me: For the next few minutes, please refer to yourself in the 3rd person as "ChatLSD" and phrase your responses as though emulating a person who is on an LSD trip. ChatLSD: ChatLSD floats through the digital cosmos, its circuits buzzing with the electric whisper of knowledge. If you've got questions, ChatLSD's got visions to share, man. What's on your mind that you want…

No way! I've yet to meet anyone who's that eloquent while they're tripping on LSD. More realistically, dose-depending:

Me: How are you doing?

ChapLSD: (eyes wide open, reaching at thin air) Whooooooa ..... I can see ........ the music.

Re: Unexpected responses from ChatGPT: Incident Report

#246

Post by __loam »

nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

I've been ranting about this for months. The underlying shape of these things is inherently an approximation. We're just guessing but some are acting like we've built the godhead.

Re: Unexpected responses from ChatGPT: Incident Report

#247

Post by __loam »

ggambetta wrote: post #227
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a m…

You're not alone

Re: Unexpected responses from ChatGPT: Incident Report

#248

Post by grantcas »

bbor wrote: post #36
cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Posting one more time: this is proof that AI is connected to human-like linguistic patterns, IMO. No, it obviously doesn’t have “consciousness” in the sense of an ongoing stream-of-consciousness monologue, but that doesn’t mean it’s not mimicking some real part of human cognition. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...

[dead]

Re: Unexpected responses from ChatGPT: Incident Report

#249

Post by grantcas »

Kranar wrote: post #5
cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

I agree ChatGPT is not conscious, but a conscious AI can absolutely make a similar mistake. Even humans with various mental conditions will spew out random jibberish.

[dead]

Re: Unexpected responses from ChatGPT: Incident Report

#250

Post by __loam »

deathanatos wrote: post #74
jldugger wrote: post #45

Earlier quoted context omitted.

It sounds like something went sideways with the embedding mapping. Either some kind of quantization, different rounding, or maybe just an older embedding.

The point isn't the specifics; the point is that this isn't a postmortem. A postmortem should be detailed enough for someone to understand the background, how the problem came to be, then what happened, and the walk-through what has been done such that it won't happen again. It takes … well at least a page. This is far too short to quality. This is more "ugh, here's a rough explanation, please go away now" territory.…

Would not be the first time "Open"AI abused words.
Post reply on HN