Isn't the fact that we are even having the debate if GPT can potentially "know what it's talking about", the real breakthrough?
GPT-3 has no idea what it’s talking about
131–140 of 323 posts
Re: GPT-3 has no idea what it’s talking about
#132Earlier quoted context omitted.
I think it would be fair to say that either outcome would be an understandable continuation of the story.
Overfitting to the edge case is missing the point. I remind readers of this continuation: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, whi…
To be fair to the AI, stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation.
Re: GPT-3 has no idea what it’s talking about
#133Just priming an immediate availability response is likely going to get poor results.
On the other hand, this does bring up an important point, which is that few people have been systemically trying to figure out how to get it to reason through problems. For instance, if you try the pure completion on WiC you get 50% chance (like in the paper) but if you improve the prompt to self-context stuff you raise it to almost 70% (http://gptprompts.wikidot.com/linguistics:word-in-context).
Re: GPT-3 has no idea what it’s talking about
#134Earlier quoted context omitted.
GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.
About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.
This is the thesis of Bender 2020 https://www.aclweb.org/anthology/2020.acl-main.463.pdf when evaluating GPT-2. They distinguish between meaning and form.
We don't _really_ know what the physical manifestations of meaning and form are in the brain... they're just concepts we invented.
If anything, GPT-3 is suggesting that either:
1. Tasks which were previously thought to require meaning actually turn out only to require form.
2. Meaning and form are more related than previously thought.
Both are interesting findings imo, but 2 would be huge, especially if it suggests how the brain might work. Could meaning be an emergent phenomenon of form?
Re: GPT-3 has no idea what it’s talking about
#135Earlier quoted context omitted.
> It's a little bit like some sort of Chinese room, or asking a non-developer to answer you programming questions by looking like something that vaguely resembles your prompt and then picking the most upvoted answer on stackoverflow. Except this isn't how it works. We know it can't be, because GPT-3 can do simple math, despite math being vastly harder with GPT-3's byte pair encoding (it doesn't use base-N, but some a…
>because GPT-3 can do simple math It can't actually, and again this is an example of the same issue. This was discussed earlier here[1]. Sometimes it produces correct arithmetic results on addition or subtraction of very small numbers, but again this is likely simply an artifact of training data. On virtually everything else it's accuracy drops to guesswork, and it doesn't even consistently get operations right that…
GPT-3's failure at larger addition sizes is almost fully due to BPE, which is incredibly pathological (392 is a ‘digit’, 393 is not; GPT-3 is also never told about the BPE scheme). When using commas, GPT-3 does OK at larger sizes. Not perfect, but certainly better than should be expected of it, given how bad BPEs are.
Re: GPT-3 has no idea what it’s talking about
#136The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…
I don't see the point of any of this, be that the original article or your points. The idea that there is a likely continuation of the story based on a few sentences as a prompt meanders between useless and wrong. It's useless because nobody wants to hear the most likely continuation of a story. It's also wrong because there is no most likely continuation without having a plot in mind. A good writer could invent almost any continuation for any of the example stories and could make it convincing and interesting.
To ask another way: What's the application of completing "stories" like that? What is the ability of doing it supposed to show?
I don't want to sound defeatist, maybe I'm really missing the point, but to me this has no more to do with Artifical Intelligence than the Hidden Markov Chain story gobblers from the 80s.
Re: GPT-3 has no idea what it’s talking about
#137The article is of course right but also a bit silly. Language models like GPT-X are producing grammatically correct sentences, along the lines of "Colorless green ideas sleep furiously". The NLP research more or less solved the old syntax problem using 'distributional semantics' but 'semantics' is a misnomer, it's all about syntax.
In fact the most useful part of the article for me is that they mentioned Douglas Summers-Stay, who does some interesting work on 'common sense' engineering, combining syntax engines like GPT-3 with knowledge graphs. https://sci-hub.tw/https://www.sciencedirect.com/science/art...
My bet is that actual AI will come from combination of these statistics-driven syntax generators with graphical causality models. Treating syntax as a kind of lower level substrate, akin to sensory modalities in vision, with intelligence model as a directed causal graph linking concepts at different levels of abstraction/chunking.
As a side note it’s funny that people working on artificial intelligence at OpenAI and elsewhere are mostly computer scientists, not cognitive psychologists or neuroscientists who might actually have a clue how intelligence works. This probably explains the proliferation of ‘backpropagation’ as primary method of artificial learning. These people are just naturally good at calculus in high school, so it’s a hammer that found its proverbial nail.
Re: GPT-3 has no idea what it’s talking about
#138I'm getting impatient with criticisms of ML models that are already covered in the papers introducing the models. OP is basically trying to get it to do what the GPT3 paper calls zero-shot inference. In the paper, it's pretty bad at zero shot inference across the board. And given what it does and how it was trained, that's unsurprising. And the point they're trying to make (that it can fail spectacularly) is also covered in the paper.
It can do cool shit. It sucks at a lot of stuff. It's impressive and limited, but the hype train seems to only allow "it's nearly human level" or "it's awful." To everybody who is arguing about its capabilities without having read the paper yet, please read it. Then we can discuss stuff that hasn't already been covered more rigorously in the original paper. I don't know Davis, but I respect Marcus, and it seems like he's pushing back on the hype more than the actual model. Just not in a way that you couldn't glean from the paper itself (it almost always sucks on zero-shot), making it pretty disingenuous. Further, from the paper [0]:
> it does little better than chance when evaluated one-shot or even few-shot on some “comparison” tasks, such as determining if two words are used the same way in a sentence, or if one sentence implies another (WIC and ANLI respectively), as well as on a subset of reading comprehension tasks.
Maybe that's the curse of doing a thing that has broad implications. You can't fit the implications in a 10 page paper, so you write a 75 page paper. The blogosphere reads the first 10 pages (if even that), and because there's so much more to it that that introduction, they go on to argue about the rest of the implications without reading it. I'm sure Marcus and Davis have read it, but this criticism wouldn't be on the front page if the rest of everyone interested in this article had read the paper too.
[0] Language Models are Few-Shot Learners https://arxiv.org/abs/2005.14165
Re: GPT-3 has no idea what it’s talking about
#139Earlier quoted context omitted.
I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.
>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it a…
> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts.
Well then you can justify it outputting anything at all.
Re: GPT-3 has no idea what it’s talking about
#140This is basically true, but I think they underrate the improvements between GPT-2 and GPT-3. My mental model is, every once in a while these systems degenerate into surreal non sequitur nonsense. GPT-3 just does it a lot less than GPT-2. It still isn’t good enough to consistently answer casual questions in a human way, but the failure rate is going down, and perhaps straightforward improvements like GPT-4 will be abl…
“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do
I think this is precisely true. The difference is, I usually go days and say thousands of sentences without my words degenerating into nonsense. And usually if I say some nonsense it's because I'm sleepy or drunk, so if I'm trying hard to be not-nonsensical I can reduce my error level to under 0.1%. GPT-3's nonsense-rate is much higher. So, I think it is promising to simply scale up this model and see how much the error rate can be reduced. Perhaps a focus on simple quantitative improvement from here can lead to a much more human performance.