Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

171–180 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#171

Post by sponnath »

thomasahle wrote: post #81
ummonk wrote: post #40

Most of the objections and their counterarguments seem like either poor objections (e.g. ad hominem against the first listed author) or seem to be subsumed under point 5. It’s annoying that most of this post focuses so much effort on discussing most of the other objections when the important discussion is the one to be had in point 5: I.e. to what extent are LLMs able to reliably make use of writing code or using log…

> I’d expect a smart human to just say “that’s too much” or “that’s beyond my abilities” rather than do a best effort faulty answer)? That's what the models did. They gave the first 100 steps, then explained how it was too much to output all of it, and gave the steps one would follow to complete it. They were graded as "wrong answer" for this. --- Source: https://x.com/scaling01/status/1931783050511126954?t=ZfmpSxH..…

Didn't they start failing well before they hit token limits? I'm not sure what the point the source you linked to is trying to make.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#172

Post by math_dandy »

FINDarkside wrote: post #108
thomasahle wrote: post #96

> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If we want to get to AGI, we will have to better. I don't get this argument. The paper is about "whether RLLMs can think". If we grant "humans make these mistakes too", but also "we still require this ability in our definition of thinking", aren't we saying "thin…

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

I was hoping the accepted definition would not use humans as a baseline, rather that humans would be an (the) example of AGI.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#173

Post by Workaccount2 »

Jensson wrote: post #165
Workaccount2 wrote: post #154

Earlier quoted context omitted.

What gets me, and the author talks about it in the post, is that people will readily attribute correct answers to "its in the training set" but nobody says anything about incorrect answers that are in the training set. LLMs get stuff in the training set wrong all the time, but nobody uses it as evidence that it probably can't lean too hard on it's memorization for complex questions it does get right. It puts LLMs in…

> It puts LLMs in an impossible position; if they are right, they memorized it, if they are wrong, they cannot reason. Both of those can be true at the same time though. They memorize a lot of things, but its fuzzy and when they remember wrong they cannot fix it via reasoning.

It's more than fuzzy, they are packing exabytes, perhaps zetabytes of training data into a few terabytes. Without any reasoning ability it must be divine intervention that they ever get anything right...

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#174

Post by firesteelrain »

travisjungroth wrote: post #168
aucisson_masque wrote: post #67

Earlier quoted context omitted.

> It’s patently obvious that LLMs can reason and solve novel problems not in their training data. Would you care to tell us more ? « It’s patently obvious » is not really an argument, I could say just as well that everyone know LLM can’t resonate or think (in the way we living beings do).

Copied from a past comment of mine: I just made up this scenario and these words, so I'm sure it wasn't in the training data. Kwomps can zark but they can't plimf. Ghirns are a lot like Kwomps, but better zarkers. Plyzers have the skills the Ghirns lack. Quoning, a type of plimfing, was developed in 3985. Zhuning was developed 100 years earlier. I have an erork that needs to be plimfed. Choose one group and one metho…

It’s just a truth table. I had a hunch that it was a truth table and then I asked AI how it figured it out and it confirmed it built a truth table. Still impressive either way

* Goal: Pick (Group ∧ Method) such that Group can plimf ∧ Method is a type of plimfing

* Only one group (Plyzers) passes the "can plimf" test

* Only one method (Quoning) is definitely plimfing

Therefore, the only valid (Group ∧ Method) combo is: → (Plyzer ∧ Quoning)

Source: ChatGPT

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#175

Post by bastawhiz »

math_dandy wrote: post #172
FINDarkside wrote: post #108

Earlier quoted context omitted.

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

I was hoping the accepted definition would not use humans as a baseline, rather that humans would be an (the) example of AGI.

The A in AGI is "artificial" which sort of precludes humans from being AGI (unless you have a very unconventional belief about the origin of humans).

Since there's not really a whole lot of unique examples of general intelligence out there, humans become a pretty straightforward way to compare.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#176

Post by sponnath »

starchild3001 wrote: post #140

We built planes—critics said they weren't birds. We built submarines—critics said they weren't fish. Progress moves forward regardless. You have a choice: master these transformative tools and harness their potential, or risk being left behind by those who do. Pro tip: Endless negativity from the same voices won't help you adapt to what's coming—learning will.

Toxic positivity is also not good.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#177

Post by travisjungroth »

firesteelrain wrote: post #174
travisjungroth wrote: post #168

Earlier quoted context omitted.

Copied from a past comment of mine: I just made up this scenario and these words, so I'm sure it wasn't in the training data. Kwomps can zark but they can't plimf. Ghirns are a lot like Kwomps, but better zarkers. Plyzers have the skills the Ghirns lack. Quoning, a type of plimfing, was developed in 3985. Zhuning was developed 100 years earlier. I have an erork that needs to be plimfed. Choose one group and one metho…

It’s just a truth table. I had a hunch that it was a truth table and then I asked AI how it figured it out and it confirmed it built a truth table. Still impressive either way * Goal: Pick (Group ∧ Method) such that Group can plimf ∧ Method is a type of plimfing * Only one group (Plyzers) passes the "can plimf" test * Only one method (Quoning) is definitely plimfing Therefore, the only valid (Group ∧ Method) combo is…

So? Is the standard now that reasoning using truth tables or reasoning that can be expressed as truth tables doesn’t count?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#178

Post by Brystephor »

jes5199 wrote: post #167

I think the Apple paper is practically a hack job - the problem was set up in such a way that the reasoning models must do all of their reasoning before outputting any of their results. Imagine a human trying to solve something this way: you’d have to either memorize the entire answer before speaking or come up with a simple pattern you could do while reciting that takes significantly less brainpower - and past a cer…

Forcing reasoning is analogous to requiring a student to show their work when solving a problem if im understanding the paper correctly.

> you’d have to either memorize the entire answer before speaking or come up with a simple pattern you could do while reciting that takes significantly less brainpower

This part i dont understand. Why would coming up with an algorithm (e.g. a simple pattern) and reciting it be impossible? The paper doesnt mention the models coming up with the algorithm at all AFAIK. If the model was able to come up with the pattern required to solve the puzzles and then also execute (e.g. recite) the pattern, then that'd show understanding. However the models didn't. So if the model can answer the same question for small inputs, but not for big inputs, then doesnt that imply the model is not finding a pattern for solving the answer but is more likely pulling from memory? Like, if the model could tell you fibbonaci numbers when n=5 but not when n=10, that'd imply the numbers are memorized and the pattern for generation of numbers is not understood.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#179

Post by krackers »

travisjungroth wrote: post #177
firesteelrain wrote: post #174

Earlier quoted context omitted.

It’s just a truth table. I had a hunch that it was a truth table and then I asked AI how it figured it out and it confirmed it built a truth table. Still impressive either way * Goal: Pick (Group ∧ Method) such that Group can plimf ∧ Method is a type of plimfing * Only one group (Plyzers) passes the "can plimf" test * Only one method (Quoning) is definitely plimfing Therefore, the only valid (Group ∧ Method) combo is…

So? Is the standard now that reasoning using truth tables or reasoning that can be expressed as truth tables doesn’t count?

If anything you'd think that the neurosymbolic people would be pleased that the LLMs do in fact reason by learning circuits representing boolean logic and truth tables. In a way they were right, it's just that starting with logic and then feeding in knowledge grounded in that logic (like Cyc) seems less scalable than feeding in knowledge and letting the model infer the underlying logic.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#180

Post by Workaccount2 »

Jensson wrote: post #166
Workaccount2 wrote: post #156

Earlier quoted context omitted.

So if they have bad memory, then they must be reasoning to get the correct answer for the problems they do solve?

A clock that is right twice a day is still broken.

I think it's more fair to say a clock that is wrong twice a day is still broken...
Post reply on HN