Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

191–200 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#191

Post by xeonmc »

bastawhiz wrote: post #175
math_dandy wrote: post #172

Earlier quoted context omitted.

I was hoping the accepted definition would not use humans as a baseline, rather that humans would be an (the) example of AGI.

The A in AGI is "artificial" which sort of precludes humans from being AGI (unless you have a very unconventional belief about the origin of humans). Since there's not really a whole lot of unique examples of general intelligence out there, humans become a pretty straightforward way to compare.

> unless you have a very unconventional belief about the origin of humans

No so unconventional in many cultures.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#192

Post by theendisney »

kgeist wrote: post #138
labrador wrote: post #74

Earlier quoted context omitted.

You're assuming we're saying LLMs can't reason. That's not what we're saying. They can execute reasoning-like processes when they've seen similar patterns, but this breaks down when true novel reasoning is required. Most people do the same thing. Some poeple can come up with novel solutions to new problems, but LLMs will choke. Here's an example: Prompt: "Let's try a reasoning test. Estimate how many pianos there are…

GPT4o isn't considered an "advanced" LLM at this point. It doesn't use reasoning. I gave your prompt to o3 pro, and this is what I got without any hints: Historic shipwrecks (1850 → 1970) • ~20 000 deep water wrecks recorded since the age of steam and steel • 10 % were passenger or mail ships likely to carry a cabin class or saloon piano • 1 piano per such vessel 20 000 × 10 % × 1 ≈ 2 000 Modern container losses (197…

The correct answer is: I'm sorry, I don't have time for this.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#193

Post by bastawhiz »

xeonmc wrote: post #191
bastawhiz wrote: post #175

Earlier quoted context omitted.

The A in AGI is "artificial" which sort of precludes humans from being AGI (unless you have a very unconventional belief about the origin of humans). Since there's not really a whole lot of unique examples of general intelligence out there, humans become a pretty straightforward way to compare.

> unless you have a very unconventional belief about the origin of humans No so unconventional in many cultures.

Certainly many cultures and religions believe in some flavor of intelligent design, but you could argue that if the natural world (for what we generally regard as "the natural world") is created by the same entity or entities that created humans, that doesn't make humans artificial. Ignoring the metaphysical (souls and such) I'm struggling to think of a culture that believes the origin of humans isn't shared by the world.

In this case, I was thinking of unusual beliefs like aliens creating humans or humans appearing abruptly from an external source such as through panspermia.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#194

Post by thrwaway55 »

Workaccount2 wrote: post #154
wohoef wrote: post #52

Good article giving some critique to Apple's paper and Gary Marcus specifically. https://www.lesswrong.com/posts/5uw26uDdFbFQgKzih/beware-gen...

What gets me, and the author talks about it in the post, is that people will readily attribute correct answers to "its in the training set" but nobody says anything about incorrect answers that are in the training set. LLMs get stuff in the training set wrong all the time, but nobody uses it as evidence that it probably can't lean too hard on it's memorization for complex questions it does get right. It puts LLMs in…

Do you hypothese that they see more wrong examples then right? Why is there concern about model collapse if they are reasoning and can sort it out, why does the data even need to be scrubbed before training?

How many r's really are in Strawberry?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#195

Post by firesteelrain »

krackers wrote: post #179
travisjungroth wrote: post #177

Earlier quoted context omitted.

So? Is the standard now that reasoning using truth tables or reasoning that can be expressed as truth tables doesn’t count?

If anything you'd think that the neurosymbolic people would be pleased that the LLMs do in fact reason by learning circuits representing boolean logic and truth tables. In a way they were right, it's just that starting with logic and then feeding in knowledge grounded in that logic (like Cyc) seems less scalable than feeding in knowledge and letting the model infer the underlying logic.

Right, that’s my point. LLMs are doing pattern abstraction and in this way can mimic logic. They are not trained explicitly to do just truth tables even thought truth tables are fundamental.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#196

Post by bowsamic »

anonthrowawy wrote: post #47
bowsamic wrote: post #23

This doesn’t address the primary issue: that they had no methodology for choosing puzzles that weren’t in the training set and indeed while they claimed to have chosen puzzles that aren’t they didn’t explain why they think that. The whole point of the paper was to test LLM reasoning in untrained cases but there’s no reason to expect such puzzles to not part of the training set, and if you don’t have any way of tellin…

how could you prove that?

You couldn’t, so such a paper cannot be scientific

(Or it should not be based on that claim as a central point, which apples paper was)

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#197

Post by bowsamic »

roywiggins wrote: post #38
bowsamic wrote: post #23

This doesn’t address the primary issue: that they had no methodology for choosing puzzles that weren’t in the training set and indeed while they claimed to have chosen puzzles that aren’t they didn’t explain why they think that. The whole point of the paper was to test LLM reasoning in untrained cases but there’s no reason to expect such puzzles to not part of the training set, and if you don’t have any way of tellin…

Isn't it worse for LLMs if an LLM that has been trained on the Towers of Hanoi still can't solve it reliably?

Yes

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#198

Post by bluefirebrand »

mountainriver wrote: post #93
devwastaken wrote: post #22

Earlier quoted context omitted.

experts are often blinded by their paychecks to see how nonsense their expertise is

Not knowing the most basic things about the subject you are critiquing is utter nonsense. Defending someone who does this is even worse

I think it's pretty fair to be critical of what LLMs are producing and how they fit into the tools without necessarily understanding how they work

If you bought a chainsaw that broke when you tried to cut down a tree, then you can criticize the chainsaw without knowing how the motor on it works, right?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#199

Post by viccis »

nerdponx wrote: post #144
viccis wrote: post #136

Earlier quoted context omitted.

>I think the answer to this question is certainly "Yes". It is unequivocally "No" . A good joint distribution estimator is always by definition a posteriori and completely incapable of synthetic a priori thought.

That doesn't seem true to me at all. Let's say you fit y=c+bx+ax^2 on the domain -10,10 with 1000 data points uniformly distributed along x and with no more than 1% noise in observed y. Your model will be pretty damn good and absolutely will be able to generate "synthetic a priori" y outputs for any given x within the domain. Now let's say you didn't know the true function and had to use a neural network instead. You…

>Your model will be pretty damn good and absolutely will be able to generate "synthetic a priori" y outputs for any given x within the domain.

You don't seem to understand what synthetic a priori means. The fact that you're asking a model to generate outputs based on inputs means it's by definition a posteriori.

>You would probably still get a great result in the sense of generating "new" outputs that are not observed in the training data, as long as they are within or reasonably close to the original domain.

That's not cognition and has no epistemological grounds. You're making the assumption that better prediction of semiotic structure (of language, images, etc.) results in better ability to produce knowledge. You can't model knowledge with language alone, the logical positivists found that out to their disappointment a century or so ago.

For example, I don't think you adequately proved this statement to be true:

>they would have to in order to continue decreasing validation loss

This works if and only if the structure of knowledge lies latently beneath the structure of semiotics. In other words, if you can start identifying the "shape" of the distribution of language, you can perturb it slightly to get a new question and expect to get a new correct answer.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#200

Post by zoul »

qarl wrote: post #182
Brystephor wrote: post #178

Earlier quoted context omitted.

Forcing reasoning is analogous to requiring a student to show their work when solving a problem if im understanding the paper correctly. > you’d have to either memorize the entire answer before speaking or come up with a simple pattern you could do while reciting that takes significantly less brainpower This part i dont understand. Why would coming up with an algorithm (e.g. a simple pattern) and reciting it be impos…

> The paper doesnt mention the models coming up with the algorithm at all AFAIK. And that's because they specifically hamstrung their tests so that the LLMs were not "allowed" to generate algorithms. If you simply type "Give me the solution for Towers of Hanoi for 12 disks" into chatGPT it will happily give you the answer. It will write program to solve it, and then run that program to produce the answer. But accordi…

This is not about finding the most effective solution, it’s about showing that they “understand” the problem. Could they write the algorithm if it were not in their training set?
Post reply on HN