Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

221–230 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#221

Earlier quoted context omitted.

> I’d expect a smart human to just say “that’s too much” or “that’s beyond my abilities” rather than do a best effort faulty answer)? That's what the models did. They gave the first 100 steps, then explained how it was too much to output all of it, and gave the steps one would follow to complete it. They were graded as "wrong answer" for this. --- Source: https://x.com/scaling01/status/1931783050511126954?t=ZfmpSxH..…

Didn't they start failing well before they hit token limits? I'm not sure what the point the source you linked to is trying to make.

OP said:

> I’d expect a smart human to just say “that’s too much” or “that’s beyond my abilities” rather than do a best effort faulty answer)?

And that's what the models did.

This is a good answer from the model. Has nothing to do with token limits.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#222
post #26

Earlier quoted context omitted.

Even Sam Altman thinks we’re not at AGI yet (although of course it’s coming “soon”).

Markus has been consistently wrong over the many years predicting the (lack of) progress of the current deep learning methods. Altman has been correct so far.

[deleted]

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#223

Earlier quoted context omitted.

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

I was hoping the accepted definition would not use humans as a baseline, rather that humans would be an (the) example of AGI.

The argument of (1) doesn't really have anything to do with humans or antromorphising. We're not even discussing AGI, we're just talking about the property of "thinking".

If somebody claims "computers can't do X, hence they can't think". A valid counter argument is "humans can't do X either, but they can think."

It's not important for the rebuttal that we used humans. Just that there exists entities that don't have property X, but are able to think. This shows X is not required for our definition of "thinking".

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#224
post #26

Earlier quoted context omitted.

Even Sam Altman thinks we’re not at AGI yet (although of course it’s coming “soon”).

Markus has been consistently wrong over the many years predicting the (lack of) progress of the current deep learning methods. Altman has been correct so far.

Marcus has made some good predictions and some bad ones. That’s usually the way with people who make specific predictions — there are no prophets.

Not sure I’d agree that SA has been any more consistently right. You can easily find examples of overconfidence from him (though he rarely says anything specific enough to count as a prediction).

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#225
post #202

Earlier quoted context omitted.

The human mind is an estimator too. The fact that the human mind can think in concepts, images AND words, and then compresses that into words for transmission, wheras LLMs think directly in words, is no object. If you watch someone reach a ledge, your mind will generate, based on past experience, a probabilistic image of that person falling. Then it will tie that to the concept of problem (self-attention) and start g…

>LLMs think Quick aside here: They do not think. They estimate generative probability distributions over the token space. If there's one thing I do agree with Dijkstra on, it's that it's important not to anthropomorphize mathematical or computing concepts. As far as the rest of your comment, I generally agree. It sort of fits a Kantian view of epistemology, in which we have sensibility giving way to semiotics (we'll…

But I do not think humans think like that by default.

When I spill a drink, I don't think "gravity". That's too slow.

And I don't think humans are particularly good at that kind of rational thinking.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#226
post #202

Earlier quoted context omitted.

>LLMs think Quick aside here: They do not think. They estimate generative probability distributions over the token space. If there's one thing I do agree with Dijkstra on, it's that it's important not to anthropomorphize mathematical or computing concepts. As far as the rest of your comment, I generally agree. It sort of fits a Kantian view of epistemology, in which we have sensibility giving way to semiotics (we'll…

But I do not think humans think like that by default. When I spill a drink, I don't think "gravity". That's too slow. And I don't think humans are particularly good at that kind of rational thinking.

>When I spill a drink, I don't think "gravity". That's too slow.

I think you do, you just don't need to notice it. If you spilled it in the International Space Station, you'd probably respond differently even if you didn't have to stop and contemplate the physics of the situation.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#227
post #204

Earlier quoted context omitted.

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

Yes. I wonder if he was thinking of ASI, not AGI

ASI meaning Artificial Super Intelligence, I guess.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#228
post #202

Earlier quoted context omitted.

The human mind is an estimator too. The fact that the human mind can think in concepts, images AND words, and then compresses that into words for transmission, wheras LLMs think directly in words, is no object. If you watch someone reach a ledge, your mind will generate, based on past experience, a probabilistic image of that person falling. Then it will tie that to the concept of problem (self-attention) and start g…

>LLMs think Quick aside here: They do not think. They estimate generative probability distributions over the token space. If there's one thing I do agree with Dijkstra on, it's that it's important not to anthropomorphize mathematical or computing concepts. As far as the rest of your comment, I generally agree. It sort of fits a Kantian view of epistemology, in which we have sensibility giving way to semiotics (we'll…

> We accomplish this by forming concepts such as "ledge", "step", "person", "gravity", etc., as we experience them until they exist in our mind as purely rational concepts we can use to reason about new experiences.

So we receive inputs from the environment and cluster them into observations about concepts, and form a collection of truth statements about them. Some of them may be wrong, or apply conditionally. These are probabilistic beliefs learned a posteriori from our experiences. Then we can do some a priori thinking about them with our eyes and ears closed with minimal further input from the environment. We may generate some new truth statements that we have not thought about before (e. g. "stepping over the ledge might not cause us to fall because gravity might stop at the ledge") and assign subjective probabilities to them.

This makes the a priori seem to always depend on previous a posterioris, and simply mark the cutoff from when you stop taking environmental input into account for your reasoning within a "thinking session". Actually, you might even change your mind mid-reasoning process based on the outcome of a thought experiment you perform which you use to update your internal facts collection. This would give the a priori reasing you're currently doing an even stronger a posteriori character. To me, these observations above basically dissolve the concept of a priori thinking.

And this makes it seem like we are very much working from probabilistic models, all the time. To answer how we can know anything: If a statement's subjective probability becomes high enough, we qualify it as a fact (and may be wrong about it sometimes). But this allows us to justify other statements (validly, in ~ 1-sometimes of cases). Hopefully our world model map converges towards a useful part of the territory!

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#229

> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If we want to get to AGI, we will have to better. I don't get this argument. The paper is about "whether RLLMs can think". If we grant "humans make these mistakes too", but also "we still require this ability in our definition of thinking", aren't we saying "thin…

Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#230

Earlier quoted context omitted.

> The key is to not become a slave to them. To realize that our worth consists not in our ability to think. And that we are more than that. I cannot afford to consider whether you are right because I am a slave to capital, and therefore may as well be a slave to capital's LLMs. The same goes for you.

I am not a slave to capital. I am a slave to the harsh nature of the world. I get too hot in summer and too cold in winter. I die of hunger. I am harassed by critters of all sorts. And when my bed breaks, to keep my fragile spine from straining at night, I _want_ some trees to be cut, some mattresses to be provisioned, some designers to be provisioned etc. And capital is what gets me that, from people I will never me…

Considering capitalism is a very new phenomenon in human history, how do you think people survived and thrived for the other 248000 years? It's as ludicrous to believe that capitalism is some kind of force of nature as it is to believe kings were chosen by god.
Post reply on HN