While Gary Marcus produces nothing of any actual worth, the rest of the AI world casually marches on incrementally improving day by day, to his utter annoyance.
GPT-3 has no idea what it’s talking about
71–80 of 323 posts
Re: GPT-3 has no idea what it’s talking about
#72I don't get it. Their methodology says > These experiments are not, by any means, either a representative or a systematic sample of anything. We designed them explicitly to be difficult for current natural language processing technology. Moreover, we pre-tested them on the "AI Dungeon" game which is powered by some version of GPT-3, and we excluded those for which "AI Dungeon" gave reasonable answers. (We did not kee…
And it also suffers from the tired assumption that GPT-3 (or any language models) should, or are designed to in any way, give reasonable answers[1]. All GPT-3 does is give likely continuations , given the training corpus. The prompts here are too short, and it could likely just be writing mediocre fiction continuations. Fiction tends to not be reasonable much of the time (to create story conflict). > "To understand w…
The question is how you know you’ve found a prompt that will give reliable results, at least for questions in some domain? Although it’s possible such a prompt exists, it’s not clear that anyone has found one, and based on architecture, there is some reason to expect that apparently-promising prompts will prove unreliable if investigated seriously.
It’s fine to do some exploratory experiments to find a good prompt, but this needs to be followed up with a separate, rigorous evaluation phase with a fixed prompt. If you keep changing the prompt then you will never know what you have and can easily be fooled by randomness.
Re: GPT-3 has no idea what it’s talking about
#73Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020
No, those concrete tests are mostly issues that researchers have been talking about for years, meaning that many of them appear on the Internet somewhere. Increasing the volume of training data to hundreds of gigabytes likely meant that the exact questions and answers appeared in the training data.
So GPT-3 didn't "smash them", it cut and pasted the answer from its training.
Re: GPT-3 has no idea what it’s talking about
#74There's been so much drift in what people expect from a language model. We used to expect a language model would tell you which sentences were likely and which were unlikely, which things were grammatical and which were not -- but this wasn't initially expected to be tied to a detailed knowledge of the world or general reasoning ability.
With GPT3, we've seen people prompt it to generate tables of factual information (e.g. state populations), and commenters can simultaneously be surprised that some of the facts are on the right scale, and also disappointed that they're wrong. Here, an AI researcher has to argue that a model trained only on text hasn't learned about physics or geometry or social norms or a bunch of other stuff that we wouldn't assume is well captured in just whatever text is available.
I think maybe the fault is not that GPT3 doesn't know these things. The fault is that as humans, we're so dependent on language both for communication with others and also for our own cognition, that when we encounter a really good language model, it's hard for us to _not_ see some glimmer of general AI. We're so impressed that we unreasonably move the goal posts. https://www.smbc-comics.com/comic/ball
And it's worth asking -- we consider a human speaker to "know" a language when they've internalized its grammar and vocabulary, but not all that specific stuff about the world. An 18th century English speaker and a 21st century English speaker were/are aware of drastically different facts, and are likely to produce different sentences, but there's something about English that they both know. Not as a criticism of GPT3 but as a question about NLP researchers -- why can we not isolate and represent that in a model?
Re: GPT-3 has no idea what it’s talking about
#75Why must we keep having this argument?
If you do research in the field you know full well that GPT/any other transformer or Bert model is generating text by regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt. The neurophysiological concept of “understanding” as most understand it is orthogonal to the way the algorithm actually works.
A more useful conversation to have might be: what sort of prompts does GPT struggle with? How might we alter the algorithm to ameliorate these issues? But instead we separate into cults of believers and nonbelievers and uselessly wax poetic about it.
Re: GPT-3 has no idea what it’s talking about
#76Re: GPT-3 has no idea what it’s talking about
#77GPT-3 was trained on internet texts, not causal/logical-reasoning only texts. Without context, there is a good chance that samples will match the distribution it was trained on.
This is a non-result, posing as something critical or important. These conclusions are obvious given the model and a basic knowledge of statistics/the transformer architecture.
A bit shameful for someone to ride on the anti-hype wave like this, I'd hope there'd be a more balanced/scientific approach to analyzing legitimate weaknesses rather than setting up strawmen then claiming victory.
Re: GPT-3 has no idea what it’s talking about
#78Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020
>GPT-3 smashed them. which isn't surprising because virtually all of the questions are so simple they could literally appear in the training data that GPT-3 was trained on. I'm a little tired of proving how "intelligent" GPT is by asking these superficial questions. the MIT article gives much better examples that actually require physical, biological or higher-level reasoning and it produces complete nonsense as one…
I think the intent was not to prove GPT-3 intelligent but rather to prove the shallowness of the critiques of it it.
Re: GPT-3 has no idea what it’s talking about
#79The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…
I stopped reading right after that clothes comment to comment exactly what you had.
If you even provide the simplest context of question answer gpt3 answers reasonably
[Prompt]
Q: What is the day after Tuesday?
A: Wednesday
Q: Yesterday I dropped my clothes off at the dry cleaner’s and I have yet to pick them up. Where are my clothes?
A: [gpt3] A: They are in the dryer.
Another give away that the article wouldn't be in good faith is that weird rant in the beginning about how OpenAI didn't share with them research access.
I think people don't talk enough about useful prompts and most demos don't bother sharing their prompt. I think people thinking about building businesses off gpt3 see their prompt as essentially their secret sauce. And maybe other tuning parameters, but there really aren't too many. You can turn up the temperature and maybe build a model to score the response or fine tune the model.
Re: GPT-3 has no idea what it’s talking about
#80This article reads like a fucking joke.