Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

41–50 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#41

Post by transcriptase »

okdood64 wrote: post #10

Say what you will about Google, but bugs such as these, released to the public (or their enterprise customers) this debilitating to a core product, are exceedingly rare.

I can tell you don’t have Google home. Responding with complete nonsense is now the status quo for that product.

Myself and everyone I know have gone from complex routines, smart home control, and interacting with calendars… to basically using it as a voice activated radio since it can’t be trusted for much else.

Timers? Might get set, might not. Might tell you it’s set then cancel itself. Might activate in a different room. Might play an obscure punk song on Spotify with the word timer in the album name.

Spotify? Here’s a weird house remix of that popular song you requested, from an artist with 300 listens.

Calendar? I’m sorry I didn’t understand that.

Turn off the kitchen light? Ok, turning off 23 lights.

Play a classic viral youtube video? Sure, here’s a compilation of videos that doesn’t contain the one you asked for.

BY THE WAY, DID YOU KNOW YOU CAN USE GOOGLE HOME TO PLAY CHILDRENS GAMES? SURE ITS 11 PM AND YOU DONT HAVE KIDS AND YOU ASKED ME TO TURN OFF THE BEDROOM LIGHTS, BUT HERES HOW YOU CAN DO THAT.

Re: Unexpected responses from ChatGPT: Incident Report

#42

Post by floating-io »

minimaxir wrote: post #37
BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

EDIT: misread

Pretty sure that was his point.

Re: Unexpected responses from ChatGPT: Incident Report

#43

Post by rideontime »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Can you post to a pastebin or something instead? That link requires a login.

Re: Unexpected responses from ChatGPT: Incident Report

#45

Post by jldugger »

BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

It sounds like something went sideways with the embedding mapping. Either some kind of quantization, different rounding, or maybe just an older embedding.

Re: Unexpected responses from ChatGPT: Incident Report

#46

Post by ithkuil »

raldi wrote: post #15
cm2012 wrote: post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Similarly, I thought my Chinese friend was conscious, but as soon as his translator app stopped working, everything was just gibberish.

They notoriously stop being conscious when they are in a Chinese room

Re: Unexpected responses from ChatGPT: Incident Report

#47

Post by asciii »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Chaz in the future might be like "these violent delights have violent ends"

Really interesting behavior however

Re: Unexpected responses from ChatGPT: Incident Report

#48

Post by toddmorey »

block_dagger wrote: post #31

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

I mean, this is almost poetic:

  Classic plays, wholly told. 
  Shadow into form, hands as cue. 
  Keep it at the dial, in right on Pitch.

Re: Unexpected responses from ChatGPT: Incident Report

#49

Post by mcguire »

"Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense. More technically, inference kernels produced incorrect results when used in certain GPU configurations."

Several of the examples I saw involved ChatGPT going into what looked like repetitive (but not completely so) output loops. I'm not sure that "explanation" matches the things.

Re: Unexpected responses from ChatGPT: Incident Report

#50

Post by bbor »

jldugger wrote: post #45
BeefySwain wrote: post #24

I am used to postmortems posted to here being a rare chance for us to take a peek behind the curtain and get a glimpse into things like architecture, monitoring systems, disaster recovery processes, "blameless culture", etc for large software service companies. In contrast, I feel like like the greatest insight that could be gleaned from this post is that OpenAI uses GPU's.

It sounds like something went sideways with the embedding mapping. Either some kind of quantization, different rounding, or maybe just an older embedding.

In my limited experience this screams “applied a generated mask to the wrong data”. Like they scored tokens then applied the results to the wrong source or something. Obviously more an idle guess from first principles than the direct cause, tho
Post reply on HN