Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

141–150 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#141

Post by jjaksic »

lossolo wrote: post #45
Jabrov wrote: post #35

Earlier quoted context omitted.

I’m so tired of hearing this be repeated, like the whole “LLMs are _just_ parrots” thing. It’s patently obvious to me that LLMs can reason and solve novel problems not in their training data. You can test this out in so many ways, and there’s so many examples out there. ______________ Edit for responders, instead of replying to each: We obviously have to define what we mean by "reasoning" and "solving novel problems"…

They can't create anything novel and it's patently obvious if you understand how they're implemented. But I'm just some anonymous guy on HN, so maybe this time I will just cite the opinion of the DeepMind CEO, who said in a recent interview with The Verge (available on YouTube) that LLMs based on transformers can't create anything truly novel.

Since when is reasoning synonymous with invention? All humans with a functioning brain can reason, but only a tiny fraction have or will ever invent anything.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#142

Post by FINDarkside »

ummonk wrote: post #121
FINDarkside wrote: post #99

Earlier quoted context omitted.

I don't think most of the objections are poor at all apart from 3, it's this article that seems to make lots of strawmans. Especially the first objection is often heard because people claim "this paper proves LLMs don't reason". The author moves goalposts and is arguing against about whether LLMs lead to AGI, which is already a strawman for those arguments. And in addition, he even seems to misunderstand AGI, thinkin…

I'm more saying that points 1 and 2 get subsumed under point 5 - to the extent that existing algorithms / logical systems for solving such problems are written by humans, an AGI wouldn't need to match the performance of those algorithms / logical systems - it would merely need to be able to create / use such algorithms and systems itself. You make a good point though that the question of whether LLMs reason or not sh…

Right, I agree there. Also that's something LLMs can already do. If you give the problem to ChatGPT o3 model, it will actually write python code, run it and give you the solution. But I think points 1 and 2 are still very valid things to talk about, because while Tower of Hanoi can be solved by writing code that doesn't apply to every problem that would require extensive reasoning.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#144

Post by nerdponx »

viccis wrote: post #136
serbuvlad wrote: post #124

Earlier quoted context omitted.

"Have we created machines that can do something qualitatevely similar to that part of us that can correlate known information and pattern recognition to produce new ideas and solutions to problems -- that part we call thinking?" I think the answer to this question is certainly "Yes". I think the reason people deny this is because it was just laughably easy in retrospect. In mid-2022 people were like. "Wow this GPT3 t…

>I think the answer to this question is certainly "Yes". It is unequivocally "No" . A good joint distribution estimator is always by definition a posteriori and completely incapable of synthetic a priori thought.

That doesn't seem true to me at all. Let's say you fit y=c+bx+ax^2 on the domain -10,10 with 1000 data points uniformly distributed along x and with no more than 1% noise in observed y. Your model will be pretty damn good and absolutely will be able to generate "synthetic a priori" y outputs for any given x within the domain.

Now let's say you didn't know the true function and had to use a neural network instead. You would probably still get a great result in the sense of generating "new" outputs that are not observed in the training data, as long as they are within or reasonably close to the original domain.

LLMs are that. With enough data and enough parameters and the right inductive bias and the right RLHF procedure etc, they are getting increasingly good at estimating a conditional next token distribution given the context. If by "synthetic" you mean that an LLM can never generate a truly new idea that was not in it's training data, then that becomes the question of what the "domain" of the data really is.

I'm not convinced that LLMs are strictly limited to ideas that they have "learned" in their data. Before LLMs, I don't think people realized just how much pattern and structure there was in human thought, and how exposed it was through text. Given the advances of the last couple of years, I'm starting to come around to the idea that text contains enough instances of reasoning and thinking that these models might develop some kind of ability to do something like reasoning and thinking simply because they would have to in order to continue decreasing validation loss.

I want to be clear that I am not at all an AI maximalist, and the fact that these things are built largely on copyright infringement continues to disgust me, as do the growing economic and environmental externalities and other problems surrounding their use and abuse. But I don't think it does any good to pretend these things are dumber than they are, or to assume that the next AI winter is right around the corner.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#145

Post by Jabrov »

labrador wrote: post #106
Jabrov wrote: post #88

Earlier quoted context omitted.

That seems like a totally reasonable response ... ?

I think you missed the part where I had to give them hinits to solve it. All 3 initially couldn't or refused saying it was not a real problem on their first try.

You must be on the wrong side of an A/B test or very unlucky.

Because I gave your exact prompt to o3, Gemini, and Claude and they all produced reasonable answers like above on the first shot, with no hints, multiple times.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#146

Post by sieabahlpark »

dang wrote: post #77
mentalgear wrote: post #29

AI hype-bros like to complain that real AI experts are too much concerned about debunking current AI then improving it - but the truth is that debunking bad AI IS improving AI. Science is a process of trial and error which only works by continuously questioning the current state.

Can you please make your substantive points without name-calling or swipes? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html .

[dead]

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#147

Post by diego898 »

DanAtC wrote: post #122
wohoef wrote: post #52

Good article giving some critique to Apple's paper and Gary Marcus specifically. https://www.lesswrong.com/posts/5uw26uDdFbFQgKzih/beware-gen...

[flagged]

Sorry - what do you mean by yud-cult? Searching google didn’t help me (as far as I can tell) - I view LW from an outside perspective as well, but don’t understand the reference

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#148

Post by f33d5173 »

diego898 wrote: post #147
DanAtC wrote: post #122

Earlier quoted context omitted.

[flagged]

Sorry - what do you mean by yud-cult? Searching google didn’t help me (as far as I can tell) - I view LW from an outside perspective as well, but don’t understand the reference

Elezier yudkowsky, often referred to as yud, started lw.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#149

Post by yen223 »

kgeist wrote: post #138
labrador wrote: post #74

Earlier quoted context omitted.

You're assuming we're saying LLMs can't reason. That's not what we're saying. They can execute reasoning-like processes when they've seen similar patterns, but this breaks down when true novel reasoning is required. Most people do the same thing. Some poeple can come up with novel solutions to new problems, but LLMs will choke. Here's an example: Prompt: "Let's try a reasoning test. Estimate how many pianos there are…

GPT4o isn't considered an "advanced" LLM at this point. It doesn't use reasoning. I gave your prompt to o3 pro, and this is what I got without any hints: Historic shipwrecks (1850 → 1970) • ~20 000 deep water wrecks recorded since the age of steam and steel • 10 % were passenger or mail ships likely to carry a cabin class or saloon piano • 1 piano per such vessel 20 000 × 10 % × 1 ≈ 2 000 Modern container losses (197…

The difference between o3 and o4-mini is so substantial I think this is the reason why people can't agree on how capable LLMs are nowadays.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#150

Post by lexh »

diego898 wrote: post #147
DanAtC wrote: post #122

Earlier quoted context omitted.

[flagged]

Sorry - what do you mean by yud-cult? Searching google didn’t help me (as far as I can tell) - I view LW from an outside perspective as well, but don’t understand the reference

Use the debatably intelligent machines for this sort of question not Google.

It seems “Yud” here is a shorthand for Yudkowsky. Hinted by the capitalization.

Post reply on HN