Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

171–180 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#171

Post by bjackman »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I don't a the problem with this communication, except they just shouldn't call it a "postmortem".

In a medical setting this is equivalent to saying "the patient was coughing up blood, and now they're dead". It's relaying some potentially useful info, and I guess it's _technically_ a post-mortem analysis but it doesn't fit expectations for the phrase!

Re: Unexpected responses from ChatGPT: Incident Report

#172

Post by shever73 »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

That last paragraph is wild! Chaz went into full Samuel Beckett mode.

Slightly off-topic, but one thing I’ve found consistently sends ChatGPT into a tailspin is asking it to analyse and solve a cryptic crossword clue. You get pages of musings, which is just a long way of saying “I don’t know”.

Re: Unexpected responses from ChatGPT: Incident Report

#173

Post by [deleted] »

ummonk wrote: post #124
cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Humans do the same thing when they get a stroke. Does that mean they don't have actual consciousness?

[deleted]

Re: Unexpected responses from ChatGPT: Incident Report

#174

Post by tomas789 »

bjackman wrote: post #171
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I don't a the problem with this communication, except they just shouldn't call it a "postmortem". In a medical setting this is equivalent to saying "the patient was coughing up blood, and now they're dead". It's relaying some potentially useful info, and I guess it's _technically_ a post-mortem analysis but it doesn't fit expectations for the phrase!

It is an industry standard word by now.

Re: Unexpected responses from ChatGPT: Incident Report

#176

Post by stravant »

__MatrixMan__ wrote: post #66
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

There are surely reasonable ways to smoke test changes to the extent that they would catch the issue that came up here.

E.g.: Have a gauntlet of 20 moderate complexity questions with machine checkable characteristics in the answer. A couple may fail incidentally now and then but if more than N/20 fail you know something's probably gone wrong.

Re: Unexpected responses from ChatGPT: Incident Report

#177

Post by gtirloni »

bjackman wrote: post #171
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I don't a the problem with this communication, except they just shouldn't call it a "postmortem". In a medical setting this is equivalent to saying "the patient was coughing up blood, and now they're dead". It's relaying some potentially useful info, and I guess it's _technically_ a post-mortem analysis but it doesn't fit expectations for the phrase!

https://sre.google/sre-book/postmortem-culture/

Re: Unexpected responses from ChatGPT: Incident Report

#178

Post by jorticka »

Rastonbury wrote: post #165
laborcontract wrote: post #62

Earlier quoted context omitted.

The explanation is definitely unsatisfying. If I had to guess it was an issue with quantization. OpenAI moves so quickly with their product and doesn't seem to be shy about quickly pushing changes to prod. There are too many times to count that, using ChatGPT, I've seen the frontend changing before my eyes or seen responses change as backend modifications are pushed. On one hand it's refreshing to see their lack of a…

I find that the most annoying and borderline unethical, hiding changes and nerfs behind the fact the output is probalistic especially when I'm paying the same amount for the product and others are paying for and using it in production. As many have mentioned, GPT-4 ability to code is not the same as 6-8 months ago

So stop paying then.

Re: Unexpected responses from ChatGPT: Incident Report

#179

Post by mbel »

tomas789 wrote: post #174
bjackman wrote: post #171

Earlier quoted context omitted.

I don't a the problem with this communication, except they just shouldn't call it a "postmortem". In a medical setting this is equivalent to saying "the patient was coughing up blood, and now they're dead". It's relaying some potentially useful info, and I guess it's _technically_ a post-mortem analysis but it doesn't fit expectations for the phrase!

It is an industry standard word by now.

Standard word which implies some level of analysis into causes of the incident. This brief note doesn’t live up to industry practice either.

Re: Unexpected responses from ChatGPT: Incident Report

#180

Post by bjackman »

gtirloni wrote: post #177
bjackman wrote: post #171

Earlier quoted context omitted.

I don't a the problem with this communication, except they just shouldn't call it a "postmortem". In a medical setting this is equivalent to saying "the patient was coughing up blood, and now they're dead". It's relaying some potentially useful info, and I guess it's _technically_ a post-mortem analysis but it doesn't fit expectations for the phrase!

https://sre.google/sre-book/postmortem-culture/

Yeah that's what I'm referring to. This isn't a postmortem as described in that book! This is just "it was broken, and then we fixed it". Which is fine, it's good to acknowledge that. Just don't call it a PM!
Post reply on HN