Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

Post reply on HN
Print view View original

251–260 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#251

Post by __loam »

kweingar wrote: post #28
swyx wrote: post #8

Earlier quoted context omitted.

bad argument that i'm very tired of. some might say that current/former world leaders also exhibit this property. not getting political but just because "math fucked up sometimes produces bad results" does not invalidate the idea that consciousness can emerge from a pile of biological or digital neurons.

I’m tired of the extremely tenuous analogies that tie computer bugs to human behavior. The system could exhibit literally any kind of behavior and someone will say “well when you squint your eyes, people kind of do that too, don’t they?”

It's so obnoxious to see people talk about us like the brain is a machine that is well understood.

Re: Unexpected responses from ChatGPT: Incident Report

#252

Post by swyx »

throwaway920102 wrote: post #129

Some samples I found quickly scanning the ChatGPT subreddit for anyone curious: https://www.reddit.com/r/ChatGPT/comments/1awm8kv/chatgpt_sp... ^ quick non sequiturs https://www.reddit.com/r/ChatGPT/comments/1aw1o9x/i_had_a_re... https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd.... ^ incredibly trippy text that devolves into strangely formatted comma separated text that almost resembles some code / amateur…

great samples recap

Re: Unexpected responses from ChatGPT: Incident Report

#254

Post by throwuwu »

nottorp wrote: post #234
joeythedolphin wrote: post #225

Earlier quoted context omitted.

I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.

"teach you all". So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.

Original sources aren’t necessarily infallible either, but that’s beside the point, asking ChatGPT needs to be viewed as asking a person to recount something from memory. It hasn’t been reviewed or edited so it’s like a first draft rather than a published document. That’s the error that people tend to make since they treat it like a search engine instead of a conversation.

Re: Unexpected responses from ChatGPT: Incident Report

#255

Post by logicprog »

ggambetta wrote: post #227
nf3 wrote: post #144

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a m…

I feel the same way. It's frustrating feeling like I'm one of the few people that can see that the emperor has no clothes, and nice to see the occasional person like you that pushes back on this cargo cult fanaticism.

Re: Unexpected responses from ChatGPT: Incident Report

#256

Post by nottorp »

throwuwu wrote: post #254
nottorp wrote: post #234

Earlier quoted context omitted.

"teach you all". So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.

Original sources aren’t necessarily infallible either, but that’s beside the point, asking ChatGPT needs to be viewed as asking a person to recount something from memory. It hasn’t been reviewed or edited so it’s like a first draft rather than a published document. That’s the error that people tend to make since they treat it like a search engine instead of a conversation.

> they treat it like a search engine instead of a conversation.

That's because web searches are so polluted that for a lot of purposes even ChatGPT 3.5 is better.

Re: Unexpected responses from ChatGPT: Incident Report

#257

Post by skissane »

card_zero wrote: post #223
skissane wrote: post #196

Earlier quoted context omitted.

> It does however seem to stop when the brain is destroyed, which makes it unlikely to be present in fundamental particles, or in parts of a disintegrated brain Does consciousness really "seem to stop" when the brain is destroyed? We can't directly observe anyone's consciousness other than our own. It is true that, when people die, we cease to have access to the outward signs we use to infer they are conscious – but…

But I also have other organs, such as my stomach, and arbitrary lumps of matter such as my elbow and my bicycle, none of which are sufficient to maintain the outward signs of consciousness after the brain is destroyed (or merely dosed with gin). So consciousness, which apparently resided in the brain and then went away when the brain was disrupted, didn't go to any of those places. Similarly, when my stomach ceases t…

> And why make this claim about consciousness alone, out of all the functions that things in the world have?

Because consciousness is fundamental in a way none of those things in the world are. It is only through our own consciousness that we can know those other things exist, indeed that we can ever know anything at all. Treating consciousness as just a thing among things ignores that it is essential, foundational, to all other things as we know them, as we can know them.

Re: Unexpected responses from ChatGPT: Incident Report

#258

Post by pests »

__MatrixMan__ wrote: post #66
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

We know more about NNs then you think. You are vastly underestimating the field and what knowledge we have.

Re: Unexpected responses from ChatGPT: Incident Report

#259

Post by mock-possum »

__MatrixMan__ wrote: post #66
montroser wrote: post #26

This explanation feels unsatisfying. It's so high-level that it's mostly void of any actual information. What was the wrong assumption that the code made that caused this wrong behavior? Why was it not caught in the many layers of automated testing before it made its way to production? What process and procedural changes are being implemented to reduce the risk of this class of bug happening again? Presumably all of…

I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…

“We don’t understand why neural networks work” is a myth. It’s not a miracle, it’s just code, and you can step through it to debug it the same way you would any other program.

Re: Unexpected responses from ChatGPT: Incident Report

#260

Post by astrange »

EchoChamberMan wrote: post #239
astrange wrote: post #118

Earlier quoted context omitted.

The LLM doesn't have secrets or other users' chats in it. Why would they put that in there?

There are historical cases of just such a thing happening. Google warns you not to put important or private information in.

That's because they have human evaluators read user prompts to decide where to make quality improvements.
Post reply on HN