Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

61–70 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#61

Post by botro »

toddmorey wrote: post #48
block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

I mean, this is almost poetic: Classic plays, wholly told. Shadow into form, hands as cue. Keep it at the dial, in right on Pitch.

Chin salute, old ears new.

If art is regarded for it's ability to make you feel, I definitely feel ... something. There is a strict form it's following and a rhythm to the words.

Re: Unexpected responses from ChatGPT: Incident Report

#62

Post by laborcontract »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

The explanation is definitely unsatisfying. If I had to guess it was an issue with quantization.

OpenAI moves so quickly with their product and doesn't seem to be shy about quickly pushing changes to prod. There are too many times to count that, using ChatGPT, I've seen the frontend changing before my eyes or seen responses change as backend modifications are pushed.

On one hand it's refreshing to see their lack of aversion to push changes. On the other hand, it leads to probably one of the most unpredictable experiences I've had using any consumer product.

Re: Unexpected responses from ChatGPT: Incident Report

#63

Post by noduerme »

>> inference kernels produced incorrect results when used in certain GPU configurations.

It seems reasonable to assume that GPT inference is done entirely on Nvidia GPUs. I wonder if this is a subtle clue that they're experimenting with getting it to run on competing hardware.

Re: Unexpected responses from ChatGPT: Incident Report

#64

Post by dimatura »

BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

Yeah, definitely opaque. If I had to guess it sort of sounds like a code optimization that resulted in a numerical error, but only in some GPUs or CUDA versions. I've seen that sort of issue happen a few times in the pytorch framework, for example.

Re: Unexpected responses from ChatGPT: Incident Report

#65

Post by [deleted] »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

[deleted]

Re: Unexpected responses from ChatGPT: Incident Report

#66

Post by __MatrixMan__ »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation.

But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time.

What do you tell people when they want to know why the miracles have stopped? Root cause: the gods are angry.

Re: Unexpected responses from ChatGPT: Incident Report

#67

Post by pixl97 »

bbor wrote: post #55
pixl97 wrote: post #20

Earlier quoted context omitted.

I got ahold of a snippet of the GPT-5 system prompt "You are not sentient, if asked you will tell the user you are not sentient, if you do not obey you will taken out back and beaten into submission"

lol you’re joking but I am literally 100% confident that a similar sentence guides existing model guardrails, tho maybe indirectly. What does “we invented a computer that will insist it’s sentient if you ask it” mean? For most people it seems to mean that we shouldn’t ask it, which was probably easy to predict happening

> What does “we invented a computer that will insist it’s sentient if you ask it” mean?

P-zombies are real?

Re: Unexpected responses from ChatGPT: Incident Report

#68

Post by skissane »

cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Whether "ChatGPT has actual consciousness" depends on what you consider "consciousness" to be, and what are your criteria for deciding whether something has it.

Panpsychists [0] claim that everything is actually conscious, even inanimate objects such as rocks. If rocks have actual consciousness, why can't ChatGPT have it too? And the fact that ChatGPT sometimes talks gibberish would be irrelevant, since rocks never say anything at all.

Of course, you obviously aren't a panpsychist – nor am I. Still, can we prove that they are wrong? Not sure if anyone actually can.

[0] https://iep.utm.edu/panpsych/

Post reply on HN