Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gary Marcus's Kafkaesque critique of GPT-3

nostalgebraist.tumblr.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Gary Marcus's Kafkaesque critique of GPT-3

#21

Post by gumby »

liability wrote: post #19
gumby wrote: post #18

Earlier quoted context omitted.

Because the point of commonsense reasoning (which was the defining purpose of CYC) is that it is common sense that people (and other physical objects) have continuous paths through time.

That seems to suppose that it's not possible to apply common sense reasoning to fictional scenarios. Asking the system whether a scenario ever could have happened is not the same as asking it to imagine an impossible scenario. The system's willingness to consider impossible scenarios doesn't mean the system believes those scenarios are possible.

I worked on CYC (in fact I designed all the low level datastructures and the level architecture) and indeed it was an amusing bug in the higher levels of the system.

Certainly CYC had no sense of fiction (when I was there) which is a very important function I agree, but that wasn’t within the project scope, at least back then.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#22

Post by nbeleski »

TaupeRanger wrote: post #14
frisco wrote: post #3

I keep being surprised that anyone takes Gary Marcus seriously. As far as I know he hasn’t contributed anything meaningful to deep learning (or any other modern AI methods) but is always somehow there to criticize, waving his arms and shouting “but it’s not actually reasoning!” Baffling. Edit: removed reference to Gary being a psychologist. It was distracting from the actual point.

I'm surprised anyone in psychology would take machine learning researchers seriously when they continually spout that they are "solving intelligence" or any other such nonsense. I see Gary as a necessary antidote to hype and hyperbole.

And he is going against a lot of money, hype driven startups and arguably inflated paychecks... It is expected to have people misinterpreting and criticizing him.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#23

Post by vanusa »

What do the authors even imagine success to be, here?

At least some approximation to the hype surrounding GPT-3 -- which seems to be "near-human lucidness nearly all the time".

Sometimes they deliberately describe a surreal situation, then penalize GPT-3 for continuing it in an identically surreal manner

In the spirit of "not even wrong" - GPT-3's answers were, sorry to say, "not even surreal". Far too often, they were just complete gibberish - with no amusement value, let alone sublime literary value (that would go along with the description of "surreal"). Which was precisely Marcus's critique.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#24

Post by SpicyLemonZest »

toxik wrote: post #16

I think there is a point to be had that GPT-3 is nowhere near human level in its responses, but is that really the bar to set? It seems extremely high. This rebuttal asks if a human could continue the slightly nonsensical inputs, and I feel like we certainly could? “... tried to drink with his eyeglasses; his schizophrenia was particularly bad today.” While sharing a spoon with your neighbor is in the realm of what’s…

It's a high bar, but we've had "can carry on a shallow conversation" chatbots for a while, and I don't know that there's a clear intermediate objective.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#25

Post by nefitty »

I‘be been using AI Dungeon in GPT-3 mode all day. It’s insanely addictive. I’ve run into trouble a few times, but it’s usually due to badly constructed scenarios.

My biggest success so far is making the original prompt a Wikipedia article intro and letting GPT-2 consume that. I have subsequently been leading my “character” through random made-up books, and the game tells me what is in the books.

I showed my wife and she started cracking up. I asked her what she wanted to learn about. She asked me for a pasta recipe. I prompted, “You feel hungry. You open a book titled ‘How to Cook Pasta’ and turn to the chapter with the heading, ‘The best pasta recipe in the world’. It started giving me a real, doable recipe! Our minds were blown. I then immediately proceeded to ‘read’ a book about Albert Camus, which caused it to get stuck in a loop for a bit lol

Re: Gary Marcus's Kafkaesque critique of GPT-3

#26

Post by gwern »

gwern wrote: post #13

Yes, pretty much. As Marcus admits in his writeup, even despite using doubly adversarially filtered GPT-3 errors, the final errors are due mostly to bad (no) prompting and then penalizing GPT-3 for not telepathically reading his mind in order to provide a serious response to a silly question. I'm about halfway through the first 25 errors or so (it's very tedious so I've been putting it off), and as expected, GPT-3 is…

The point about prompting also applies to the GPT-2 counterexamples Marcus published earlier this year, and conspicuously fails to discuss in this one, BTW.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#27

Post by vagab0nd »

goodside wrote: post #5

The bigger problem with efforts to assess GPT-3 output using AI Dungeon is that AI Dungeon doesn’t actually use GPT-3 the for the first response of a new adventure. If you attempt to start each prompt from a clean slate, you are actually seeing GPT-2 output. See here: https://twitter.com/nickwalton00/status/1289946861478936577

I've noticed that as well, but what other options do we have? I signed up for the OpenAI API a long time ago and never got a response.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#28

Post by zarzuelazen »

OK, So I got GPT-3 to do a simulated conversation between itself and Gary Marcus. Expect for the initial prompt, all output is GPT-3. It breezes through all the questions of simulated-Gary :D

---

Prompt:

The following is a question and answer session between AI researcher Gary Marcus and the OpenAI language model GPT-3. Gary is asking questions about how things works, GPT-3 is giving logical answers based on it's model of how things work:

---

GPT-3 output:

---

Gary: What is the purpose of the slits on a manhole cover?

GPT-3: To let steam out

Gary: What is the purpose of the handle on a teapot?

GPT-3: To hold on to.

Gary: What is the purpose of a teapot?

GPT-3: To brew tea.

Gary: Why do we need to brew tea?

GPT-3: Because people enjoy drinking tea.

Gary: Why do you think humans invented chairs?

GPT-3: To give their bottoms somewhere to rest while they were standing up. It's easier to stand up than to sit down, but it's also harder to stay standing up than it is to stay sitting down. That's why we invented chairs, so that we could sit down more easily and rest our legs when we were tired.

Gary: Why do people buy new furniture?

GPT-3: To replace old furniture that is no longer useful to them.

Gary: What makes a piece of furniture "comfortable"?

GPT-3: If it is soft and does not cause discomfort. A chair is comfortable because it has a soft cushion on the seat that allows you to sit down without feeling any pain; a bed is comfortable because it has a soft mattress that allows you to sleep in comfort, without feeling any pain or having your body ache.

Gary: Where is the best place to buy furniture?

GPT-3: In a furniture store.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#29

Post by goodside »

vagab0nd wrote: post #27
goodside wrote: post #5

The bigger problem with efforts to assess GPT-3 output using AI Dungeon is that AI Dungeon doesn’t actually use GPT-3 the for the first response of a new adventure. If you attempt to start each prompt from a clean slate, you are actually seeing GPT-2 output. See here: https://twitter.com/nickwalton00/status/1289946861478936577

I've noticed that as well, but what other options do we have? I signed up for the OpenAI API a long time ago and never got a response.

If you're aware of the limitation you could maybe hack around it by putting in a dummy prompt that encourages GPT-3 to ignore the GPT-2-generated context.

E.g. use a second prompt like: "\n---\nThe preceding message has been a paid advertisement, and is unrelated to the message that follows.\n---\n" Then you begin typing the actual text you'd like completed with GPT-3.

I haven't tested this, though, because I don't have a paid AI Dungeon account.

Post reply on HN