Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

Post reply on HN
Print view View original

301–310 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#301

Post by RugnirViking »

sigotirandolas wrote: post #265
Workaccount2 wrote: post #255

Earlier quoted context omitted.

People aren't claiming that they are holding textbooks in their model, that would just be even more evidence of reasoning (the LLM would have to reason what textbook to reference, and then extrapolate from the textbook(s) how to solve the problem at hand - pretty much what students in school do; study the textbook and reason from it to answer new test questions) People are claiming that the models sit on a vast archi…

I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). And this only appears intelligent because this pattern matching is really good and backed by a huge amount of compressed/memorized answers. The AI pessimist's argument is that there's a h…

I think I hadn't seen the ai pessimist side laid out like that before,but i feel like i understand it a lot better now. Definitely worth a good think at least. The difference between reproducing an algorithm you've seen before Vs ongoing following the steps required and course correcting, showing understanding. I feel like this can be seen in other high profile failures both in AIs and in humans.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#302

Post by RugnirViking »

neoden wrote: post #152

> Puzzles a child can do Certainly, I couldn't solve Hanoi's towers with 8 disks purely in my mind without being able to write down the state of every step or having a physical state in front of me. Are we comparing apples to apples?

Why without writing down each step? Would you be able to solve it writing each step required in sequence? Thinking between each one? Pretty sure i could, isn't that closer to an LLM?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#303

Post by RugnirViking »

woodturner550 wrote: post #298

As we are losing our rights in America(won't even acknowledge the 'new knowledge'), this becomes important to freedom loving people of the world. Please, acknowledge this important work for the world. This 'new knowledge' is free to the world. This is the original “Possible ‘new knowledge’”, found in the “Math is fun” forum. All files can be found at: https://drive.google.com/drive/folders/1wpd5-2-4SZkZka284sbp... Ma…

You seem like a smart person. Get yourself checked for schizophrenia. This is filled with hallmarks. It happens to the best of us, and can be treated.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#304

Post by RugnirViking »

barrkel wrote: post #235
charcircuit wrote: post #110

Earlier quoted context omitted.

>with tool use A LLM with tool use can solve anything. It is interesting to try and measure its capabilities without tools.

I don't think the first is true at all, unless you imagine some powerful oracle tools. I think the second is interesting for comparing models, but not interesting for determining the limits of what models can automate in practice. It's the prospect of automating labour which makes AI exciting and revolutionary, not their ability when arbitrarily restricted.

Isn't the point of automating labour though to automate that which is not/was already not automated?

It would draw on many previously written examples of algorithms to write the code for solving Hanoi. To solve a novel problem with tool use, one needs to work sequentially while staying on task, notice where you've gone wrong, and backtrack.

I don't want to overstate the case here, I'm sure there is work where there's enough intersection between previously existing stuff in the dataset and few enough sequential steps required that useful work can be done, but idk how much you've tried using this stuff as a labour saving device, there's less low hanging fruit than one might think, but more than zero.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#305

Post by tim333 »

thomasahle wrote: post #96

> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If we want to get to AGI, we will have to better. I don't get this argument. The paper is about "whether RLLMs can think". If we grant "humans make these mistakes too", but also "we still require this ability in our definition of thinking", aren't we saying "thin…

To understand the argument you have to allow that the Gary Marcus neural network has a fixed bias towards LLMs being rubbish and so tends to confabulate these sort of things. (Geoffrey Hinton is quite funny on that https://youtu.be/d7ltNiRrDHQ)

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#306

Post by godelski »

YeGoblynQueenne wrote: post #293
godelski wrote: post #275

Earlier quoted context omitted.

> That is wishful thinking popularised by Ilya Sutskever Ilya and Hinton have claimed even crazier things | to understand next token prediction you must understand the casual reality This is objectively false. It's a result known in physics to be wrong for centuries. You can probably reason a weaker case yourself, that I'm sure you can make accurate predictions about some things without fully understanding them. But…

Hinton and Sutskever are victims of their own success: they can say whatever they like and nobody dares criticise them, or tell them how they're wrong. I recently watched a video of Sutskever speaking to some students, not sure where and I can't dig out the link now. To summarise he told them that the human brain is a biological computer. He repeated this a couple of times then said that this is why we can create a d…

I'm fine with calling the brain a computer. A computer is a very vague term. But yes, I agree that the conclusion does not necessarily follow. It's possible, not not necessarily

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#307

Post by godelski »

jsnell wrote: post #295
godelski wrote: post #286

Earlier quoted context omitted.

I'm replying to this > the model printing out a bunch of steps, saying there is no point in doing it thousands of times more.

Ok, but the very next sentence was: > And they they'd either output an the algorithm for printing the rest in words or code. So clearly you already knew that your strawman was not relevant. Why try it anyway?

Because that wasn't the task given to them. It's like giving a student a test and you asking them to solve an equation and they give you the general form. It's incomplete

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#308

Post by g42gregory »

We don't know what intelligence is, we don't know what thinking is, and we don't know what reasoning is.

How can we then assess if machine is doing it?

As Demis Hassabis put it a while back: we are building AI [partially] to understand how our own brain works.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#309

Post by adastra22 »

RugnirViking wrote: post #299
adastra22 wrote: post #215

Earlier quoted context omitted.

Yes, but “good at” here has a very limited, technical meaning, which can be oversimplified as “better than random chance.” If something can be better than random chance in any arbitrary problem domain it was not trained on, that is AGI.

That raises the plausible question; are there problem domains where humans cannot do better than random chance, given repeated attempts?

Bitcoin mining.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#310

Post by amohn9 »

chongli wrote: post #242
amohn9 wrote: post #114

Earlier quoted context omitted.

It might not fit your work, but there are tons of areas where “good enough” can still provide a lot of value. I’m sure you’d be thrilled with a tool that could correctly tell you if Apple’s stock was going up or down tomorrow 70% of the time.

I work in a mail room sending hard copy letters to customers. If I got my job right only 70% of the time then I’d be causing massive privacy breaches daily by sending the wrong personal information to the wrong customers. Would you trust an AI that gets your banking transactions right only 70% of the time?

No. I also wouldn’t use a hammer to cut a board in half - I’d grab a saw. Knowing how to pick the right tool is a fundamental part of being a good engineer. Sometimes 70% is unacceptable, sometimes it’s exceptional. LLMs are incredible technology, but also just another tool in the toolbox. Use them where they fit, not where they don’t.
Post reply on HN