Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

231–240 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#231

Post by pfsalter »

mavamaarten wrote: post #213
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

Honestly, I don't think we're entitled to a deep technical write-up. We are users of their product, they messed something up, and that's it. Unless they signed an SLA agreement, it's just shit that happens. They don't have to give a technical view into their internal kitchen if they don't want to.

The biggest problem is that if you're using this in production then it could cause problems. It's not like a traditional outage where you get a load of 503 errors, the system appears to be working correctly but generates gibberish. If you're using this in a chat bot you could be liable for things it's saying...

Re: Unexpected responses from ChatGPT: Incident Report

#232

Post by Zetobal »

montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

FP is not deterministic across GPU generations. They basically say with a lot of mambo jambo that they deployed code on the wrong GPU architecture.

Re: Unexpected responses from ChatGPT: Incident Report

#233

Post by EchoChamberMan »

FridgeSeal wrote: post #98
__MatrixMan__ wrote: post #66

Earlier quoted context omitted.

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.

The way companies choose to portray themselves is a rather unreliable way of predicting their actions.

Re: Unexpected responses from ChatGPT: Incident Report

#234

Post by nottorp »

joeythedolphin wrote: post #225
nottorp wrote: post #219

Earlier quoted context omitted.

IMO you're using the wrong term or have a low bar for 'intelligence'. It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't. That may be intelligence but it doesn't justify the religious like tone some people use when talking ab…

I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.

"teach you all".

So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.

Re: Unexpected responses from ChatGPT: Incident Report

#235

Post by EchoChamberMan »

ggambetta wrote: post #227
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a m…

Sales people, and company owners, have a vested interest in pushing the narrative that the illusion is actually magic.

Re: Unexpected responses from ChatGPT: Incident Report

#236

Post by EchoChamberMan »

joeythedolphin wrote: post #225
nottorp wrote: post #219

Earlier quoted context omitted.

IMO you're using the wrong term or have a low bar for 'intelligence'. It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't. That may be intelligence but it doesn't justify the religious like tone some people use when talking ab…

I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.

It is, unquestionably, from the math, a "word gargler" as you put it.

https://xkcd.com/1838/

Re: Unexpected responses from ChatGPT: Incident Report

#238

Post by EchoChamberMan »

ralfd wrote: post #190

Was all chatGPT affected? If one uses chatGPT for enterprise and build a product feature around it, is that version freezed or did it also return garbage?

Not all, I don't think. There are fixed and variable models. An enterprise product? Depends on how it was built. If you have a team, you should be asking them, not strangers here, and expect this kind of behavior from AI in the future, these things are incredibly new.

> not production ready. not applicable for enterprise. still fancy snake oil. [1]

[1] https://news.ycombinator.com/item?id=39463505

Re: Unexpected responses from ChatGPT: Incident Report

#239

Post by EchoChamberMan »

astrange wrote: post #118
_heimdall wrote: post #84

This is exactly the kind of issue that can lead to unintended consequences. What if, instead of spewing out seemingly nonsense answers, the LLM spewed out very real answers that violated built-in moderation protocols? Or shared secrets or other users chats? What if a bug released accidentally stumbled upon how to allow the LLM to become self aware? Or paranoid? These potentials seem outlandish, but we honestly don't…

The LLM doesn't have secrets or other users' chats in it. Why would they put that in there?

There are historical cases of just such a thing happening. Google warns you not to put important or private information in.

Re: Unexpected responses from ChatGPT: Incident Report

#240

Post by bongodongobob »

skissane wrote: post #193
bongodongobob wrote: post #156

Earlier quoted context omitted.

I think it would be more fitting to say a rock, a grain of sand, or an atom is the least conscious thing. Pansychism isn't trying to claim it can think or anything like that. If that's your takeaway I think you are misunderstanding the analogy. A rock exists in the world and exerts it's "will" granted to it by the laws of physics. So rolling falling etc is how it "interacts" with its world. You can think of our brain…

> I think it would be more fitting to say a rock, a grain of sand, or an atom is the least conscious thing. Pansychism isn't trying to claim it can think or anything like that. If that's your takeaway I think you are misunderstanding the analogy. No, I don't think I've misunderstood anything; on the contrary, I think you are mixing up thought and consciousness. Thought is just one type of consciousness among many; to…

I don't disagree.
Post reply on HN