Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

21–30 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#21

Post by avsteele »

foldr wrote: post #14
avsteele wrote: post #10

This doesn't rebut anything from the best critique of the Apple paper. https://arxiv.org/abs/2506.09250

It does rebut point (1) of the abstract. Perhaps not convincingly, in your view, but it does directly addresses this kind of response.

Papers make specific conclusions based on specific data. The paper I linked specifically rebuts the conclusions of the paper. Gary makes vague statements that could be interpreted as being related.

It is scientific malpractice to write a post supposedly rebutting responses to a paper and not directly address the most salient one.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#22

Post by devwastaken »

mountainriver wrote: post #17
bluefirebrand wrote: post #2

I'm glad to read articles like this one, because I think it is important that we pour some water on the hype cycle If we want to get serious about using these new AI tools then we need to come out of the clouds and get real about their capabilities Are they impressive? Sure. Useful? Yes probably in a lot of cases But we cannot continue the hype this way, it doesn't serve anyone except the people who are financially i…

I’ll take critiques from someone who knows what a test train split is. The idea that a guy so removed from machine learning has something relevant to say about its capabilities really speaks to the state of AI fear

experts are often blinded by their paychecks to see how nonsense their expertise is

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#23

Post by bowsamic »

This doesn’t address the primary issue: that they had no methodology for choosing puzzles that weren’t in the training set and indeed while they claimed to have chosen puzzles that aren’t they didn’t explain why they think that. The whole point of the paper was to test LLM reasoning in untrained cases but there’s no reason to expect such puzzles to not part of the training set, and if you don’t have any way of telling if it is not or then your paper is not going to work out

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#24

Post by kadushka »

foldr wrote: post #18
astrange wrote: post #12

Earlier quoted context omitted.

Gary Marcus always, always says AI doesn't actually work - it's his whole thing. If he's posted a correct argument it's a coincidence. I remember seeing him claim real long-time AI researchers like David Chapman (who's a critic himself) were wrong anytime they say anything positive. (em-dash avoided to look less AI) Of course, the main issue with the field is the critics /should/ be correct. Like, LLMs shouldn't work…

I don’t read GM as saying that LLMs “don’t work” in a practical sense. He acknowledges that they have useful applications. Indeed, if they didn’t work at all, why would he be advocating for regulating their use? He just doesn’t think they’re close to AGI.

The funny thing is, if you asked “what is AGI” 5 years ago, most people would describe something like o3.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#25

Post by senko »

bluefirebrand wrote: post #2

I'm glad to read articles like this one, because I think it is important that we pour some water on the hype cycle If we want to get serious about using these new AI tools then we need to come out of the clouds and get real about their capabilities Are they impressive? Sure. Useful? Yes probably in a lot of cases But we cannot continue the hype this way, it doesn't serve anyone except the people who are financially i…

Gary Marcus isn't about "getting real", it's making a name for himself as a contrarian to the popular AI narrative.

This article may seem reasonable, but here he's defending a paper that in his previous article he called "A knockout blow for LLMs".

Many of his articles seem reasonable (if a bit off) until you read a couple dozen a spot a trend.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#26

Post by foldr »

kadushka wrote: post #24
foldr wrote: post #18

Earlier quoted context omitted.

I don’t read GM as saying that LLMs “don’t work” in a practical sense. He acknowledges that they have useful applications. Indeed, if they didn’t work at all, why would he be advocating for regulating their use? He just doesn’t think they’re close to AGI.

The funny thing is, if you asked “what is AGI” 5 years ago, most people would describe something like o3.

Even Sam Altman thinks we’re not at AGI yet (although of course it’s coming “soon”).

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#27

Post by bigyabai »

bluefirebrand wrote: post #2

I'm glad to read articles like this one, because I think it is important that we pour some water on the hype cycle If we want to get serious about using these new AI tools then we need to come out of the clouds and get real about their capabilities Are they impressive? Sure. Useful? Yes probably in a lot of cases But we cannot continue the hype this way, it doesn't serve anyone except the people who are financially i…

There's something innately funny about "HN's undying optimism" and "bad-news paper from Apple" reaching a head like this. An unstoppable object is careening towards an impervious wall, anything could happen.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#28

Post by DiogenesKynikos »

bluefirebrand wrote: post #2

I'm glad to read articles like this one, because I think it is important that we pour some water on the hype cycle If we want to get serious about using these new AI tools then we need to come out of the clouds and get real about their capabilities Are they impressive? Sure. Useful? Yes probably in a lot of cases But we cannot continue the hype this way, it doesn't serve anyone except the people who are financially i…

I don't understand what people mean when they say that AI is being hyped.

AI is at the point where you can have a conversation with it about almost anything, and it will answer more intelligently than 90% of people. That's incredibly impressive, and normal people don't need to be sold on it. They're just naturally impressed by it.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#29

Post by mentalgear »

AI hype-bros like to complain that real AI experts are too much concerned about debunking current AI then improving it - but the truth is that debunking bad AI IS improving AI. Science is a process of trial and error which only works by continuously questioning the current state.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#30

Post by marvinborner »

hiddencost wrote: post #3

Why do we keep posting stuff from Gary? He's been wrong for decades but he keeps writing this stuff. As far as I can tell he's the person that people reach for when they want to justify their beliefs. But surely being this wrong for this wrong should eventually lead to losing ones status as an expert.

Is this supposed to be a joke reflecting point (3)?
Post reply on HN