Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gary Marcus's Kafkaesque critique of GPT-3

nostalgebraist.tumblr.com

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Gary Marcus's Kafkaesque critique of GPT-3

#11

Post by liability »

tomcam wrote: post #10
liability wrote: post #7

Earlier quoted context omitted.

If his background were in selling used cars, would his opinion receive any traction at all? I think it's fair to question the relevance of somebody's background if their background is the root of their supposed relevance.

Or you could, you know, evaluate things on the technical merits

You mean like the rebuttal the article gives? There is little more to Gary Marcus's position to rebut because his position is so weak. If he had a more relevant background then perhaps his opinions would have more substance and, consequently, warrant more substantial responses.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#12

Post by visarga »

jszymborski wrote: post #4
frisco wrote: post #3

I keep being surprised that anyone takes Gary Marcus seriously. As far as I know he hasn’t contributed anything meaningful to deep learning (or any other modern AI methods) but is always somehow there to criticize, waving his arms and shouting “but it’s not actually reasoning!” Baffling. Edit: removed reference to Gary being a psychologist. It was distracting from the actual point.

While I don't necessarily agree with a lot of Gary Marcus' arguments re: AI, I don't think it's fair to gate-keep based on what you perceive his background is. AI is an incredibly multi-disciplinary field, and it's a little silly to imply that people coming from a psychology background cannot contribute meaningfully to the discussion.

It's not that he is a psychologist, it's that he doesn't have any idea about what he's talking about in ML. It's important to have results when you want to change other people's opinions, he has only opinions. Let him do better than GPT-3 and show us how it's done.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#13

Post by gwern »

Yes, pretty much. As Marcus admits in his writeup, even despite using doubly adversarially filtered GPT-3 errors, the final errors are due mostly to bad (no) prompting and then penalizing GPT-3 for not telepathically reading his mind in order to provide a serious response to a silly question.

I'm about halfway through the first 25 errors or so (it's very tedious so I've been putting it off), and as expected, GPT-3 is often correct zero-shot if you use even a trivial short prompt like 'This is a series of scenarios describing a human taking actions in the world, designed to test physical and common-sense reasoning.'

Re: Gary Marcus's Kafkaesque critique of GPT-3

#14

Post by TaupeRanger »

frisco wrote: post #3

I keep being surprised that anyone takes Gary Marcus seriously. As far as I know he hasn’t contributed anything meaningful to deep learning (or any other modern AI methods) but is always somehow there to criticize, waving his arms and shouting “but it’s not actually reasoning!” Baffling. Edit: removed reference to Gary being a psychologist. It was distracting from the actual point.

I'm surprised anyone in psychology would take machine learning researchers seriously when they continually spout that they are "solving intelligence" or any other such nonsense. I see Gary as a necessary antidote to hype and hyperbole.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#16

Post by toxik »

I think there is a point to be had that GPT-3 is nowhere near human level in its responses, but is that really the bar to set? It seems extremely high.

This rebuttal asks if a human could continue the slightly nonsensical inputs, and I feel like we certainly could? “... tried to drink with his eyeglasses; his schizophrenia was particularly bad today.” While sharing a spoon with your neighbor is in the realm of what’s possible, it does seem very odd, seeing as there is no prior relationship established to this neighbor. I also wonder what would’ve happened if an actual neighbor had been mentioned before, i.e. a door neighbor. I think any human would then read neighbor as next door neighbor. Would the network pick up on this and specify seat neighbor?

Re: Gary Marcus's Kafkaesque critique of GPT-3

#17

Post by liability »

gumby wrote: post #15

The “two Dead artists” photo was famously a problem for CYC too (could Socrates and Einstein have met? Sure: I don’t know how they spent their whole lives, so some of it could have overlapped.)

Why shouldn't Socrates and Einstein meet? IIRC they did in Bill and Ted. Having an 'AI dungeon master' entertain fictional scenarios hardly debunks anything.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#18

Post by gumby »

liability wrote: post #17
gumby wrote: post #15

The “two Dead artists” photo was famously a problem for CYC too (could Socrates and Einstein have met? Sure: I don’t know how they spent their whole lives, so some of it could have overlapped.)

Why shouldn't Socrates and Einstein meet? IIRC they did in Bill and Ted. Having an 'AI dungeon master' entertain fictional scenarios hardly debunks anything.

Because the point of commonsense reasoning (which was the defining purpose of CYC) is that it is common sense that people (and other physical objects) have continuous paths through time.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#19

Post by liability »

gumby wrote: post #18
liability wrote: post #17

Earlier quoted context omitted.

Why shouldn't Socrates and Einstein meet? IIRC they did in Bill and Ted. Having an 'AI dungeon master' entertain fictional scenarios hardly debunks anything.

Because the point of commonsense reasoning (which was the defining purpose of CYC) is that it is common sense that people (and other physical objects) have continuous paths through time.

That seems to suppose that it's not possible to apply common sense reasoning to fictional scenarios. Asking the system whether a scenario ever could have happened is not the same as asking it to imagine an impossible scenario. The system's willingness to consider impossible scenarios doesn't mean the system believes those scenarios are possible.

Re: Gary Marcus's Kafkaesque critique of GPT-3

#20

Post by mellosouls »

toxik wrote: post #16

I think there is a point to be had that GPT-3 is nowhere near human level in its responses, but is that really the bar to set? It seems extremely high. This rebuttal asks if a human could continue the slightly nonsensical inputs, and I feel like we certainly could? “... tried to drink with his eyeglasses; his schizophrenia was particularly bad today.” While sharing a spoon with your neighbor is in the realm of what’s…

I think there is a point to be had that GPT-3 is nowhere near human level in its responses, but is that really the bar to set? It seems extremely high.

Unfortunately the hype surrounding GPT-3 (and deep learning before it) wrongly implies we are making great strides towards AGI; it's inevitable people then ask for the evidence and set the bar so.

Post reply on HN