Since when did incident postmortems became this watered down non-technical BS. If I'm a paying corporate customer, I would expect a much greater detailed RCA and action plan to prevent similar occurrence in the future. Publishing these postmortems is about holding yourself accountable in public in a manner that shows how thorough and seriously you take it and this does not accomplish that.
At minimum, I want a why-5 analysis. Let me start with the first question:
1. Why did ChatGPT generate gibberish?
A: the model chose slightly wrong numbers.
2. Why did the model choose slightly wrong numbers?
LLMs can have secrets if they were scraped in the training data. And how do we know definitively what is done with chat logs? The LLM model is a black box for OpenAI (they don't know what was learned or why it was learned), and OpenAI is a black box for users (we don't know what data they collect or how they use it).
You can probe what was learned if you have access to the model; it'll tell you, especially if you do it before applying the safety features. A good heuristic for whether they would train user chats into the model is whether this makes any sense. But it doesn't; it's not valuable. They could be saying anything in there, it's likely private, and it's probably not truthful information. Presumably they do do something wi…
> You can probe what was learned if you have access to the model; it'll tell you, especially if you do it before applying the safety features.
Does that involve actually parsing the data itself, or effectively asking the model questions to see what was learned?
If the data model itself can be parsed and analyzed directly by humans that is better than I realized. If its abstracted through an interpreter (I'm sure my terminology is off here) similar to the final GPT product then we still can't really see what was learned.
Parsimony leads to wrangling about which is the simplest explanation, yes. Consciousness is ill-defined, also. It does however seem to stop when the brain is destroyed, which makes it unlikely to be present in fundamental particles, or in parts of a disintegrated brain.
> It does however seem to stop when the brain is destroyed, which makes it unlikely to be present in fundamental particles, or in parts of a disintegrated brain Does consciousness really "seem to stop" when the brain is destroyed? We can't directly observe anyone's consciousness other than our own. It is true that, when people die, we cease to have access to the outward signs we use to infer they are conscious – but…
But I also have other organs, such as my stomach, and arbitrary lumps of matter such as my elbow and my bicycle, none of which are sufficient to maintain the outward signs of consciousness after the brain is destroyed (or merely dosed with gin). So consciousness, which apparently resided in the brain and then went away when the brain was disrupted, didn't go to any of those places. Similarly, when my stomach ceases to digest or my bicycle ceases to roll forward, the digestive function doesn't migrate to the brain and the rolling function isn't taken over by the elbow. So we can choose in each case between the hypothesis "physically stopped working" or "metaphysically sent its function to another mysterious place", and again I'll appeal to parsimony on this one: why should the function be transferred somewhere mysterious by a means outside of our experience and ability to explain? And why make this claim about consciousness alone, out of all the functions that things in the world have? There's a kind of fallacy going on here along the lines of "I can't fully explain what this thing is, therefore every kind of mysterious and magical supposition can be roped into its service and claim plausibility."
I think your questions all grew up in a world where the people operating the thing knew some rationalist who could think deductively about its operation. But neural networks... they're an exercise in empiricism. We only ever understood that it works, never why. It's sort of a miracle that it doesn't produce buggy output all the time. What do you tell people when they want to know why the miracles have stopped? Root c…
I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.
case in point: one of the root system prompts leaked out recently and it's pretty clear the "laziness" on a number of fronts is directly because of the root system prompt.
Aren't we also machines? If we make theories that discount our own intelligence that's just throwing our hands up and giving up. Maybe I would agree with you if LLMs weren't already embedded in a lot of useful products. I myself measurably save a lot of time using ChatGPT in such "unintelligent conversations". It's intelligent in the ways that matter for a tool.
IMO you're using the wrong term or have a low bar for 'intelligence'. It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't. That may be intelligence but it doesn't justify the religious like tone some people use when talking ab…
I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.
There are surely reasonable ways to smoke test changes to the extent that they would catch the issue that came up here. E.g.: Have a gauntlet of 20 moderate complexity questions with machine checkable characteristics in the answer. A couple may fail incidentally now and then but if more than N/20 fail you know something's probably gone wrong.
Reading between the lines a bit here, it would probably require more specialized testing infrastructure than normal. I used to be an SRE at Google and I wrote up internal postmortems there. To me, this explanation feels a lot like they are trying to avoid naming any of their technical partners, but the most likely explanation for what happened is that Microsoft installed some new GPU racks without necessarily informi…
This is the most grounded take and what I think probably happened as well.
For companies this size, with these valuations, everything the public is meant to see is heavily curated to accommodate all kinds of non-technical interests.
Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... Peop…
Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow.
I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation".
It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion.
I know I'm part of a minority of people who think this way :( The last few months have felt like I'm taking crazy pills :(
I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis) https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f... EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun. EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6…
I kind of have the feeling that someone made a simply typo "ChazGPT" in some rules or persona description, which then caused this behavior.
I’d genuinely expect the people who built and operate the thing to have a far better write up than what amounted to “it no worked lol”. Sure, NN’s are opaque, but this org paints itself as the herald and shepard of ai and they just produced a write up that’s hardly worthy of a primary-school-child’s account of their recent holiday.
case in point: one of the root system prompts leaked out recently and it's pretty clear the "laziness" on a number of fronts is directly because of the root system prompt.
I know about the leaked prompts and the laziness issues, but haven't read the prompt. What specifically about the prompt changes do you feel have led to laziness?
I don’t think it has conciousness, but your argument is not very strong, this is more akin to a sensory problem. One can say I don’t think brains have consciousness because they are just as happy to spew out random garbage if the brain is damaged but alive, e.g. aphasia where involuntary use of incorrect words occurs.
I think what they mean is, the model was unable to recognize that "it" was having issues, so it's not self-aware in that sense. We can all calm down a little, it's ok.
There are examples of it recognizing there are issues in this very thread.
Not that people always recognize such issues anyway.