Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

311–320 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#311

Post by jcz_nz »

DiogenesKynikos wrote: post #238
xienze wrote: post #236

Earlier quoted context omitted.

But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes…

LLMs are perfectly capable of writing code to solve problems that are not in their training set. I ask LLMs to write code for niche problems that you won't find answers to just by Googling all the time. The LLMs usually get it right.

Actually, my experience at least that when dealing with novel problems LLMs fail miserably. Try accessing uncommon API’s - or areas where you’re unsure an API actually exists (REST against Exchange for admin stuff!). Both ChatGPT and Claude produce nice looking solutions dependent on non-existent libraries. Repeatedly.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#312

Post by tomhow »

astrange wrote: post #279
tomhow wrote: post #220

Earlier quoted context omitted.

Please don't do this here.

I think it's pretty important to know not to literally join a cult.

I understand that, but the guidelines don't get relaxed because the topic seems important to you. It's common for people to think that a particular issue is so important that the normal rules shouldn't apply or should be interpreted differently in that case, but we can't run the site like that.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#313

Post by MoonGhost »

jltsiren wrote: post #218
MoonGhost wrote: post #211

Earlier quoted context omitted.

> The average human is useless for pretty much everything but capable of learning to perform almost any task But only the limited number of tasks per human. > Or perhaps AGI should be able to reach the level of an experienced professional in any task. Even if it performs just better than untrained human but on any task this will be superhuman level. As no human can do it.

The G in AGI stands for "general", not for "superhuman". An intelligence that can't learn to perform information processing and decision-making tasks people routinely do does not seem very general to me.

Here is the big question: should it be equal or better then every single person? If we assume that every healthy person is 'generally intelligent' then probably this is a benchmark. Because not every person can do the tasks that other persons do routinely. Probably we shouldn't demand it from AGI either. At least not from a single model. But it makes sense to request that specialized model can be created (or trained, fine tuned) for every task humans can do.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#314

Post by elbear »

godelski wrote: post #280
elbear wrote: post #277

Earlier quoted context omitted.

If we're talking about solving an equation, for example, it's not hard to memorize. Actually, that's how most students do it, they memorize the steps and what goes where[1]. But they don't really know why the algorithm works the way it does. That's what I meant by understanding. [1] In learning psychology there is something called the interleaving effect. What it says is that you solve several problems of the same ki…

Yes, tests fail in this method. But I think you can understand why the failure is larger when we're talking about a giant compression machine. It's not even a leap in logic. Maybe a small step

I'm not sure what you mean. Btw, I'm not in the field, just have thought a lot about the topic.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#315

Post by exe34 »

thomasahle wrote: post #288
exe34 wrote: post #234

Earlier quoted context omitted.

Oh we've shifted the goal post to quality now, very good! That does rest the case.

Exactly. If the paper title had been "LLMs are not that great at thinking", nobody would have had an issue.

I trust you'd have come up with something.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#316

Post by barrkel »

charcircuit wrote: post #273
barrkel wrote: post #235

Earlier quoted context omitted.

I don't think the first is true at all, unless you imagine some powerful oracle tools. I think the second is interesting for comparing models, but not interesting for determining the limits of what models can automate in practice. It's the prospect of automating labour which makes AI exciting and revolutionary, not their ability when arbitrarily restricted.

Search is already a pretty powerful oracle to defer an answer to a human and is a common tool most AI use today. What current models can automate is not what the paper was trying to answer.

What current models can automate is why they are exciting, and the attention the paper is getting because of how it cuts into this excitement. It follows logically that the attention is somewhat misplaced.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#317

Post by barrkel »

RugnirViking wrote: post #304
barrkel wrote: post #235

Earlier quoted context omitted.

I don't think the first is true at all, unless you imagine some powerful oracle tools. I think the second is interesting for comparing models, but not interesting for determining the limits of what models can automate in practice. It's the prospect of automating labour which makes AI exciting and revolutionary, not their ability when arbitrarily restricted.

Isn't the point of automating labour though to automate that which is not/was already not automated? It would draw on many previously written examples of algorithms to write the code for solving Hanoi. To solve a novel problem with tool use, one needs to work sequentially while staying on task, notice where you've gone wrong, and backtrack. I don't want to overstate the case here, I'm sure there is work where there's…

There is a decent labour savings to be had in code generation, but under strict guidance with examples.

There's a more substantial savings to be had in research scenarios. The AI can read more and synthesize more, and faster, than I can on my own, and provide references for checking correctness.

I'm not confident enough to say that the approaches being taken now have a hard stopping point any time soon or are inherently bound to a certain complexity.

Human minds can only cope with a certain complexity too and need abstraction to chunk details into atomic units following simpler rules. Yet we've come a long way with our limited ability to cope with complexity.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#318

Post by xienze »

thomasahle wrote: post #289
xienze wrote: post #236

Earlier quoted context omitted.

But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes…

> But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. The paper doesn't give any evidence humans are able to do this. And I honestly find it very implausible. Even Gary Marcus admits in (1) that humans would probably make mistakes.

You are aware that humans created and solved the puzzle in the first place, right? Not sure I understand this line of reasoning that if there are humans in this world incapable of solving some problem then boom, checkmate, LLMs can reason about and understand problems like humans do.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#319

Post by neoden »

ben-schaaf wrote: post #268
neoden wrote: post #152

> Puzzles a child can do Certainly, I couldn't solve Hanoi's towers with 8 disks purely in my mind without being able to write down the state of every step or having a physical state in front of me. Are we comparing apples to apples?

Writing things down and reading them back is quite literally the only thing LLMs do.

Generating text into the current context is not the same as writing down. It's the same as having a thought and putting it into short-term memory. An analogy for writing down would be sending something to an MCP server that provides context-independent memory functionality.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#320

Post by neoden »

RugnirViking wrote: post #302
neoden wrote: post #152

> Puzzles a child can do Certainly, I couldn't solve Hanoi's towers with 8 disks purely in my mind without being able to write down the state of every step or having a physical state in front of me. Are we comparing apples to apples?

Why without writing down each step? Would you be able to solve it writing each step required in sequence? Thinking between each one? Pretty sure i could, isn't that closer to an LLM?

I mean I need to offload state of the puzzle being solved from my brain to an external memory device — paper, in this case. Keeping that state in my mind would be much harder. It's like some people can play chess within their minds without a board, but it's obviously not something that everyone can do
Post reply on HN