Live data from Hacker News

Unexpected responses from ChatGPT: Incident Report

status.openai.com

141–150 of 277 posts

Re: Unexpected responses from ChatGPT: Incident Report

#141

Earlier quoted context omitted.

Yeah, definitely opaque. If I had to guess it sort of sounds like a code optimization that resulted in a numerical error, but only in some GPUs or CUDA versions. I've seen that sort of issue happen a few times in the pytorch framework, for example.

Yeah, definitely opaque. I wonder what the AI would say if someone asked it what happened. It would be pretty funny if it gave a detailed answer.

Someone upthread did just that. It needs some love.

https://news.ycombinator.com/item?id=39462686

Re: Unexpected responses from ChatGPT: Incident Report

#142
post #36
post #2

This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.

Posting one more time: this is proof that AI is connected to human-like linguistic patterns, IMO. No, it obviously doesn’t have “consciousness” in the sense of an ongoing stream-of-consciousness monologue, but that doesn’t mean it’s not mimicking some real part of human cognition. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...

Just a note that consciousness isn't always or even mostly a verbal "monologue", assuming you mean that. Apologies if you didn't!

A significant percentage of people never have an inner voice at all. And few people use it most of the time.

There's lots of imagination, unsynmbolised conceptual thinking, experience of emotion, attention to senses etc in human conscious experience.

See Hurlburt's Descriptive Experience Sampling for a reference!

Re: Unexpected responses from ChatGPT: Incident Report

#144
Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being.

A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure...

People are talking about ChatGPT hallucinating. I think it's rather us humans who are.

Re: Unexpected responses from ChatGPT: Incident Report

#145

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

> Chord it in, air beated fenn. Aurous to the Pan, mimic and bold. As it steep, chunns at the bun. Keive the dells, the stars to run. No wane, all gleam, in big das brund. Sky high, stout in the hale. Woods to fetch, flied in the low.

I think all that talk of music pushed Chaz into some kind of trance and it just started jamming!

Re: Unexpected responses from ChatGPT: Incident Report

#146

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…

Why is the nonsense paragraph still grammatical? I thought the tokens were beings elected more or less at random.

Re: Unexpected responses from ChatGPT: Incident Report

#147

Earlier quoted context omitted.

LLMs can have secrets if they were scraped in the training data. And how do we know definitively what is done with chat logs? The LLM model is a black box for OpenAI (they don't know what was learned or why it was learned), and OpenAI is a black box for users (we don't know what data they collect or how they use it).

You can probe what was learned if you have access to the model; it'll tell you, especially if you do it before applying the safety features. A good heuristic for whether they would train user chats into the model is whether this makes any sense. But it doesn't; it's not valuable. They could be saying anything in there, it's likely private, and it's probably not truthful information. Presumably they do do something wi…

> You can probe what was learned if you have access to the model; it'll tell you

Could that not also be hallucinated?

Re: Unexpected responses from ChatGPT: Incident Report

#150
post #147

Earlier quoted context omitted.

You can probe what was learned if you have access to the model; it'll tell you, especially if you do it before applying the safety features. A good heuristic for whether they would train user chats into the model is whether this makes any sense. But it doesn't; it's not valuable. They could be saying anything in there, it's likely private, and it's probably not truthful information. Presumably they do do something wi…

> You can probe what was learned if you have access to the model; it'll tell you Could that not also be hallucinated?

By probe, I mean observe the internal activations. There are methods that can suggest if it's hallucinating or not, and ones that can delete individual pieces of knowledge from the model.
Post reply on HN