To summarise: we spent billions to make intelligent machines and when they're asked to solve toy problems all we get is excuses.
Seven replies to the viral Apple reasoning paper and why they fall short
261–270 of 331 posts
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#262> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If we want to get to AGI, we will have to better. I don't get this argument. The paper is about "whether RLLMs can think". If we grant "humans make these mistakes too", but also "we still require this ability in our definition of thinking", aren't we saying "thin…
>> I don't get this argument.
The argument is that LLMs are computer systems and a computer system that's as bad as a human is less useful than a human.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#263> We have every right to expect machines to do things we can’t.
Not really, this makes little sense in general, but also when in comes to this specific type is machine. In general: you can have a machine that is worse than human in everything that it does yet still be immensely valuable because it's very cheap.
In this specific case:
> AGI should be a step forward
Nope, read the definition. Matching human level intelligence, warts and all, will by definition reach AGI.
> in many cases LLMs are a step backwards
That's ok, use them in cases where it's a step forward, what's the big deal?
> note the bait and switch from “we’re going to build AGI that can revolutionize the world” to “give us some credit, our systems make errors and humans do, too”.
Ah, well, again, not really, the author just has unrealistic model of the minimum requirements for a revolution.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#264Earlier quoted context omitted.
Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.
But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes…
> can be solved without "tools" by humans, simply by understanding the problem,
This already excludes a lot of humans
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#265Earlier quoted context omitted.
I'm not convinced by this argument. You can fit a bunch of books covering up to MSc level maths on less than 100MB. After that point, more books will mostly be redundant information so it doesn't need much more space for maths beyond that. Similarly TBs of Twitter/Reddit/HN add near zero new information per comment. If anything you can fit an enormous amount of information in 1MB - we just don't need to do it because…
People aren't claiming that they are holding textbooks in their model, that would just be even more evidence of reasoning (the LLM would have to reason what textbook to reference, and then extrapolate from the textbook(s) how to solve the problem at hand - pretty much what students in school do; study the textbook and reason from it to answer new test questions) People are claiming that the models sit on a vast archi…
I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). And this only appears intelligent because this pattern matching is really good and backed by a huge amount of compressed/memorized answers.
The AI pessimist's argument is that there's a huge gap between the compute required for this pattern matching, and the compute required for human level reasoning, so AGI isn't coming anytime soon.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#266Earlier quoted context omitted.
But I do not think humans think like that by default. When I spill a drink, I don't think "gravity". That's too slow. And I don't think humans are particularly good at that kind of rational thinking.
>When I spill a drink, I don't think "gravity". That's too slow. I think you do, you just don't need to notice it. If you spilled it in the International Space Station, you'd probably respond differently even if you didn't have to stop and contemplate the physics of the situation.
I think they may have been referring to the fact that in the case of a spilled drink there's a shortcut from the sensory input to a motor output. Maybe you reach for the falling cup, maybe you back away to not get spilled on. These don't really require the conscious mind at all.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#267Earlier quoted context omitted.
Because that would prove absolutely nothing. There are numerous examples of tower of Hanoi explanations in the training set.
How do you check that a human understood it and not simply memorised different approaches?
You ask them to solve several instances of the problem?
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#268> Puzzles a child can do Certainly, I couldn't solve Hanoi's towers with 8 disks purely in my mind without being able to write down the state of every step or having a physical state in front of me. Are we comparing apples to apples?
Writing things down and reading them back is quite literally the only thing LLMs do.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#269Good article giving some critique to Apple's paper and Gary Marcus specifically. https://www.lesswrong.com/posts/5uw26uDdFbFQgKzih/beware-gen...
Honest question: does the opinion of Gary Marcus still count? His criticism seems more philosophical than scientific. It's hard for me see what he builds or reasons to get to his conclusions.
I don't know but the standard reply to all of Gary Marcus' criticisms is that they don't count because it's Gary Marcus, which of course is a big honking ad-hominem.
Re: Seven replies to the viral Apple reasoning paper and why they fall short
#270Earlier quoted context omitted.
People aren't claiming that they are holding textbooks in their model, that would just be even more evidence of reasoning (the LLM would have to reason what textbook to reference, and then extrapolate from the textbook(s) how to solve the problem at hand - pretty much what students in school do; study the textbook and reason from it to answer new test questions) People are claiming that the models sit on a vast archi…
I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication). And this only appears intelligent because this pattern matching is really good and backed by a huge amount of compressed/memorized answers. The AI pessimist's argument is that there's a h…
> I think the more realistic argument is that the model can generalize, but only by learning shortcuts (e.g. how to pattern match a problem to a likely answer) and simple algorithms (e.g. how to propagate carries in a multiplication).
This is exactly what humans do too. Anything more and we need to use tools to externalize state and algorithms. Pen and paper are tools too.