Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

281–290 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#281

Post by sigotirandolas »

TeMPOraL wrote: post #270
sigotirandolas wrote: post #265

Earlier quoted context omitted.

I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). And this only appears intelligent because this pattern matching is really good and backed by a huge amount of compressed/memorized answers. The AI pessimist's argument is that there's a h…

> I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). This is exactly what humans do too. Anything more and we need to use tools to externalize state and algorithms. Pen and paper are tools too.

My thought is that we humans are bad (by computer standards) at arithmetic and memorization because those are not evolutionarily useful on their own.

On the other hand general problem solving is, and so far any attempt to replicate it using computer algorithms has more or less failed. So it must be more complex than just some simple heuristics.

Perhaps the answer is just "more compute" but the argument that "because LLMs somewhat resemble human reasoning, we must be really close!" (instead of 25+ years away) seems wishful thinking, when:

(1) LLMs leverage a much bigger knowledge base than any human can memorize, yet

(2) LLMs fail spectacularly at certain problems and behaviours humans find easy

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#282

Post by daveguy »

Workaccount2 wrote: post #173
Jensson wrote: post #165

Earlier quoted context omitted.

> It puts LLMs in an impossible position; if they are right, they memorized it, if they are wrong, they cannot reason. Both of those can be true at the same time though. They memorize a lot of things, but its fuzzy and when they remember wrong they cannot fix it via reasoning.

It's more than fuzzy, they are packing exabytes, perhaps zetabytes of training data into a few terabytes. Without any reasoning ability it must be divine intervention that they ever get anything right...

LLMs are trained with hundreds of terabytes of data to a few petabyte at most. You are off by 3 to 6 orders of magnitude in your estimate of training data. They aren't literally trained on "all the data of the internet". That would be a divergent nightmare. Catastrophic forgetting is still a problem with neural networks and ML algorithms in general. Humans are probably trained on less than half an exabyte of data given the ~1Gbps of sensory data we receive in a lifetime. That's still ~20 petabytes of data by age 5. A 400B parameter LLM with 100 examples per parameter would equal about 640 TB (F16 parameters) of training data. That's the order of magnitude of current models.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#284

Post by soulofmischief »

Zigurd wrote: post #97
Spooky23 wrote: post #58

Earlier quoted context omitted.

Remember Web 3.0? Lol

It's unfortunate that a discussion about LLM weaknesses is giving crypto bro. But telling. There are a lot of bubble valuations out there.

It's only telling of the people who have a nascent understanding of tech cycles and who are more interested in confirming biases than attempting to respect or understand a subculture, and who are unable to recognize that every hype cycle will attract parasitic undesirables who are not representative of the movement they are hijacking.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#285

Post by jsnell »

godelski wrote: post #274
jsnell wrote: post #216

Earlier quoted context omitted.

The paper doesn't mention it because either the researchers did not care to check the outputs manually, or reporting what was in the outputs would have made it obvious what their motives were. When this research has been reproduced, the "failures" on the Tower of Hanoi are the model printing out a bunch of steps, saying there is no point in doing it thousands of times more. And they they'd either output an the algori…

It's really easy to make a billion dollars. Just make a really useful app and sell it. There's no point explaining the rest since it's so trivial.

That seems like a complete non sequitur. This is the model explaining the rest. Obviously the explanation is not very interesting since the Towers of Hanoi is not an interesting problem. But that's on the researches for choosing something with a trivial algorithm if their goal was to test reasoning abilities.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#286

Post by godelski »

jsnell wrote: post #285
godelski wrote: post #274

Earlier quoted context omitted.

It's really easy to make a billion dollars. Just make a really useful app and sell it. There's no point explaining the rest since it's so trivial.

That seems like a complete non sequitur. This is the model explaining the rest. Obviously the explanation is not very interesting since the Towers of Hanoi is not an interesting problem. But that's on the researches for choosing something with a trivial algorithm if their goal was to test reasoning abilities.

I'm replying to this

  > the model printing out a bunch of steps, saying there is no point in doing it thousands of times more.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#287

Post by thomasahle »

sigotirandolas wrote: post #281
TeMPOraL wrote: post #270

Earlier quoted context omitted.

> I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). This is exactly what humans do too. Anything more and we need to use tools to externalize state and algorithms. Pen and paper are tools too.

My thought is that we humans are bad (by computer standards) at arithmetic and memorization because those are not evolutionarily useful on their own. On the other hand general problem solving is, and so far any attempt to replicate it using computer algorithms has more or less failed. So it must be more complex than just some simple heuristics. Perhaps the answer is just "more compute" but the argument that "because…

> On the other hand general problem solving is, and so far any attempt to replicate it using computer algorithms has more or less failed.

Well, this is what the whole debate is about isn't it? Can LRMs do "general problem solving"? Can humans? What exactly does it mean?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#288

Post by thomasahle »

exe34 wrote: post #234
autobodie wrote: post #111

Earlier quoted context omitted.

I rest my case — the question concerns a quality, not a quantity. These juvenile comparisons are mere excuses.

Oh we've shifted the goal post to quality now, very good! That does rest the case.

Exactly. If the paper title had been "LLMs are not that great at thinking", nobody would have had an issue.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#289

Post by thomasahle »

xienze wrote: post #236
briandw wrote: post #229

Earlier quoted context omitted.

Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.

But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes…

> But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out.

The paper doesn't give any evidence humans are able to do this. And I honestly find it very implausible. Even Gary Marcus admits in (1) that humans would probably make mistakes.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#290

Post by TeMPOraL »

YeGoblynQueenne wrote: post #260
saberience wrote: post #259

Earlier quoted context omitted.

That’s exactly what humans do though lol. We reason about things based on our training data. We have a hard time or impossible time reasoning about things we haven’t trained on. Ie: a human with no experience of board games cannot reason about chess moves. A human with no math knowledge cannot reason about math problems. How would expect an LLM to reason about something with no training data?

>> Ie: a human with no experience of board games cannot reason about chess moves. A human with no math knowledge cannot reason about math problems. Then how did the first humans solve math and chess problems, if there were none around solved to give them examples of how to solve them in the first place?

Incrementally, by tiny steps. Including a lot of doing first, then realizing later this is relevant to some chess/math thing.

Also the idea of "problems" like "chess problems" and "math problems" is itself constructed. Chess wasn't created by stacking together enough "chess problems" until they turned into a game - it was invented and tuned as a game for a long time before someone thought about distilling "problems" from it, in order to aid learning the game; from there, it also spilled out into space of logical puzzles in general.

This is true of every skill, too. You first have people who master something by experience, and then you have others who try to distill elements of that skill into "problems" or "exercise regimes" or such, in order to help others reach mastery quicker. "Problems" never come first.

Also: most "problems" are constructed around a known solution. So another answer to "how did the first humans solve" them is simply, one human back-constructed a problem around a solution, and then gave it to a friend to solve. The problem couldn't be too hard either, as it's no fun to not be able to solve it, or to require too much hints. Hence, tiny increments.

Post reply on HN