Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

81–90 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#81

Post by thomasahle »

ummonk wrote: post #40

Most of the objections and their counterarguments seem like either poor objections (e.g. ad hominem against the first listed author) or seem to be subsumed under point 5. It’s annoying that most of this post focuses so much effort on discussing most of the other objections when the important discussion is the one to be had in point 5: I.e. to what extent are LLMs able to reliably make use of writing code or using log…

> I’d expect a smart human to just say “that’s too much” or “that’s beyond my abilities” rather than do a best effort faulty answer)?

That's what the models did. They gave the first 100 steps, then explained how it was too much to output all of it, and gave the steps one would follow to complete it.

They were graded as "wrong answer" for this.

---

Source: https://x.com/scaling01/status/1931783050511126954?t=ZfmpSxH...

> If you actually look at the output of the models you will see that they don't even reason about the problem if it gets too large: "Due to the large number of moves, I'll explain the solution approach rather than listing all 32,767 moves individually"

> At least for Sonnet it doesn't try to reason through the problem once it's above ~7 disks. It will state what the problem and the algorithm to solve it and then output its solution without even thinking about individual steps.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#83

Post by neepi »

wongarsu wrote: post #71
neepi wrote: post #36

Earlier quoted context omitted.

Indeed. I completely agree with this. My objection to the whole thing is the AI hype bros, which is really the funding solicitation facade over everything rather the truth, only has one outcome and that is that it cannot be sustained. At that point all investor confidence disappears, the money is gone and everyone loses access to the tools that they suddenly built all their dependencies on because it's all proprietar…

A lot of the best internet services came around in the decade after the dot-com crash. There is a chance Anthropic or OpenAI may not survive when funding suddenly dries up, but existing open weight models won't be majorly impacted. There will always be someone willing to host DeepSeek for you if you're willing to pay. And while it will be sad to see model improvements slow down when the bubble bursts there is a lot o…

Someone might host DeepSeek for you but you'll pay through the nose for it and it'll be frozen in time because the training cost doesn't have the revenue to keep the ball rolling.

I'm not sure the GPU market won't collapse with it either. Possibly taking out a chunk of TSMC in the process, which will then have knock on effects across the whole industry.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#84

Post by senko »

steamrolled wrote: post #43
senko wrote: post #25

Earlier quoted context omitted.

Gary Marcus isn't about "getting real", it's making a name for himself as a contrarian to the popular AI narrative. This article may seem reasonable, but here he's defending a paper that in his previous article he called "A knockout blow for LLMs". Many of his articles seem reasonable (if a bit off) until you read a couple dozen a spot a trend.

> Gary Marcus isn't about "getting real", it's making a name for himself as a contrarian to the popular AI narrative. That's an odd standard. Not wanting to be wrong is a universal human instinct. By that logic, every person who ever took any position on LLMs is automatically untrustworthy. After all, they made a name for themselves by being pro- or con-. Or maybe a centrist - that's a position too. Either he makes g…

He makes many very good points:

For example he continusly calls out AGI hype for what it is, and also showcases dangers of naive use of LLMs (eg. lawyers copy-pasting hallucinated cases into their documents, etc). For this, he has plenty of material!

He also makes some very bad points and worse inferences: that LLMs as a technology are useless because they can't lead to AGI, that hallucation makes LLMs useless (but then he contradicts himself in another article conceding they "may have some use"), that because they can't follow an algorithm they're useless, etc, that scaling laws are over therefore LLMs won't advance (he's been making that for a couple of years), that AI bubble will collapse in a few months (also a few years of that), etc.

Read any of his article (I've read too many, sadly) and you'll never come to the conclusion that LLMs might be a useful technology, or be "a good thing" even in some limited way. This just doesn't fit with reality I can observe with my own eyes.

To me, this shows he's incredibly biased. That's okay if he wants to be a pundit - I couldn't blame Gruber for being biased about Apple! But Marcus presents himself as the authority on AI, a scientist, showing a real and unbiased view on the field. In fact, he's as full of hype as Sam Altman is, just in another direction.

Imagine he was talking about aviation, not AI. 787 dreamliner crashes? "I've been saying for 10 years that airplanes are unsafe, they can fall from the sky!" Boeing the company does stupid shit? "Blown door shows why airplane makers can't be trusted" Airline goes bankrupt? "Air travel winter is here"

I've spoken to too many intelligent people who read Marcus, take him at his words and have incredibly warped views on the actual potential and dangers of AI (and send me links to his latest piece with "so this sounds pretty damning, what's your take?"). He does real damage.

Compare him with Simon Willison, who also writes about AI a lot, and is vocal about its shortcomings and dangers. Reading Simon, I never get the feeling I'm being sold on a story (either positive or negative), but that I learned something.

Perhaps a Marcus is inevitable as a symptom of the Internet's immune system to the huge amount of AI hype and bullshit being thrown around. Perhaps Gary is just fed up with everything and comes out guns blazing, science be damned. I don't know.

But in my mind, he's as much BSer as the AGI singularity hypers.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#85

Post by js8 »

Gigachad wrote: post #60
2muchcoffeeman wrote: post #49

Earlier quoted context omitted.

I think it largely depends on what you’re writing. I’ve had it reply to corporate emails which is good since I need to sound professional not human. If I’m coding it still needs a lot of baby sitting and sometimes I’m much faster than it.

And then the person on the end is using AI to summarise the email back to normal English. To what end?

But look the GDP has increased!

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#86

Post by senko »

newswasboring wrote: post #69
senko wrote: post #25

Earlier quoted context omitted.

Gary Marcus isn't about "getting real", it's making a name for himself as a contrarian to the popular AI narrative. This article may seem reasonable, but here he's defending a paper that in his previous article he called "A knockout blow for LLMs". Many of his articles seem reasonable (if a bit off) until you read a couple dozen a spot a trend.

What exactly is your objection here? That the guy has an opinion and is writing about it?

Replied elsewere in the thread: https://news.ycombinator.com/item?id=44279283

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#87

Post by goalieca »

Jabrov wrote: post #35
labrador wrote: post #31

The key insight is that LLMs can 'reason' when they've seen similar solutions in training data, but this breaks down on truly novel problems. This isn't reasoning exactly, but close enough to be useful in many circumstances. Repeating solutions on demand can be handy, just like repeating facts on demand is handy. Marcus gets this right technically but focuses too much on emotional arguments rather than clear explanat…

I’m so tired of hearing this be repeated, like the whole “LLMs are _just_ parrots” thing. It’s patently obvious to me that LLMs can reason and solve novel problems not in their training data. You can test this out in so many ways, and there’s so many examples out there. ______________ Edit for responders, instead of replying to each: We obviously have to define what we mean by "reasoning" and "solving novel problems"…

So far they cannot even answer questions which are straight up fact checking and search engine like queries. Reasoning means they would be able to work through a problem and generate a proof they way a student might.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#88

Post by Jabrov »

labrador wrote: post #74
aucisson_masque wrote: post #61

Earlier quoted context omitted.

That’s the opposite of reasoning tho. Ai bros want to make people believe LLM are smart but they’re not capable of intelligence and reasoning. Reasoning mean you can take on a problem you’ve never seen before and think of innovative ways to solve it. LLM can only replicate what is in its data, it can in no way think or guess or estimate what will likely be the best solution, it can only output a solution based on a p…

You're assuming we're saying LLMs can't reason. That's not what we're saying. They can execute reasoning-like processes when they've seen similar patterns, but this breaks down when true novel reasoning is required. Most people do the same thing. Some poeple can come up with novel solutions to new problems, but LLMs will choke. Here's an example: Prompt: "Let's try a reasoning test. Estimate how many pianos there are…

That seems like a totally reasonable response ... ?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#89

Post by landl0rd »

bluefirebrand wrote: post #2

I'm glad to read articles like this one, because I think it is important that we pour some water on the hype cycle If we want to get serious about using these new AI tools then we need to come out of the clouds and get real about their capabilities Are they impressive? Sure. Useful? Yes probably in a lot of cases But we cannot continue the hype this way, it doesn't serve anyone except the people who are financially i…

[flagged]

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#90

Post by Zigurd »

spookie wrote: post #50
fhd2 wrote: post #4

Earlier quoted context omitted.

Even of the people invested in these tools, hype only benefits those attempting a pump and dump scheme, or those selling training, consulting or similar services around AI. People who try to make genuine progress, while there's more money in it now, might just have to deal with another AI winter soon at this rate.

Think the same thing, we need more breakthroughs. Until then, it is still risky to rely on AI for most applications. The sad thing is that most would take this comment the wrong way. Assuming it is just another doomer take. No, there is still a lot to do, and promissing the world too soon will only lead to disappointment.

This is the thing of it: "for most applications."

LLMs are not thinking. They way they fail, which is confidently and articulately, is one way they reveal there is no mind behind the bland but well-structured text.

But if I was tasked with finding 500 patents with weak claims or claims that have been litigated and knocked down, I would turn into LLMs to help automate that. One or two "nines" of reliability is fine, and LLMs would turn this previously impossible task into something plausible to take on.

Post reply on HN