Live data from Hacker News

Why Claude's Comment Paper Is a Poor Rebuttal

victoramartinez.com

11–20 of 77 posts

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#11
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

It may genuinely be the case that slower humans are not generally intelligent. But that sounds rather snobbish so it's not an opinion I'd like to express frequently.

I think the complaint made by apple is quite logical though and you mischaracterise it here. The question asked in the Apple study was "if I give you the algorithm that solves a puzzle, can you solve that puzzle?" The answer for most humans should be yes. Indeed, the answer is yes for computers which are not generally intelligent. Models failed to execute the algorithm. This suggests that the models are far inferior to the human mind in terms of their computational ability, which precedes general intelligence if you ask me. It seems to indicate that the models are using more of a "guess and check" approach than actually thinking. (A specifically interesting result was that model performance did not substantially improve between a puzzle with the solution algorithm given, and one where no algorithm was given.)

You can sort of imagine the human mind as the head of a Turing Machine which operates on language tokens, and the goal of an LLM is to imitate the internal logic of that head. This paper seems to demonstrate that they are not very good at doing that. It makes a lot of sense when you think about it, because the models work by consuming their entire input at once where the human mind operates with only a small working memory. A fundamental architectural difference which I suspect is the cause of the collapse noted in the Apple paper.

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#12
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

[flagged]

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#13
post #12
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

[flagged]

Because slower humans have intelligence?

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#14
post #5

Earlier quoted context omitted.

Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling. If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out. Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work eve…

>mediocre people will produce even worse work and will weed themselves out. [[Citation needed]] I don't believe anyone who has experienced working with other people - in the workspace, in school, whatever - believes that people get weeded out for mediocre output.

Weeded out to where anyway? Doing some silly thing, like being a cashier or taxi driver?

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#15
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

Someone who can reliably solve towers of Hanoi with n=4 and who has been told the algorithm should be able to do it with n=6,7,8. Don't forget, these models aren't learning how to do it from scratch the way a child might.

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#16
post #11
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

It may genuinely be the case that slower humans are not generally intelligent. But that sounds rather snobbish so it's not an opinion I'd like to express frequently. I think the complaint made by apple is quite logical though and you mischaracterise it here. The question asked in the Apple study was "if I give you the algorithm that solves a puzzle, can you solve that puzzle?" The answer for most humans should be yes…

I think a human will struggle to solve Hanoi using the recursive algorithm for even 6 disks, even given pen and paper.

Does that change if you give them the algorithm description? No. Conversely, the LLMs already know the algorithm, so including it in the prompt makes no difference.

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#18
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

Mass Effect?

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#19
> I would consider this a death blow paper to the current push for using LLMs and LRMs as the basis for AGI.

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully."

Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness."

Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

Re: Why Claude's Comment Paper Is a Poor Rebuttal

#20
post #6

Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.

People confuse performance and internal presentation.

A simple calculator is vastly better as adding numbers than any human. An chess engine will rival any human grand master. No one would say that this got us closer to AGI.

We could absolutely see LLMs that produce poetry that humans can not tell apart or even prefer to human made poetry. We could have LLMs that are perfectly able to convince humans that they have consciousness and emotions.

Would we have have achieved AGI then? Does that mean those LLMs have gotten consciousness and emotions? No.

The question of consciousness is based on what is going on in the inside, how the reasoning happening and not the output. In fact the first AGI might perform significantly worse in most tasks that current LLMs.

LLMs are extremely impressive but they are not thinking. They do not have consciousness. It might be technically impossible for them to develop anything like that or at least it would require significantly bigger models.

> where slower humans are disbarred from intelligence

Humans have value for being humans. Whether they are slow or fast at thinking. Whether they are neurodivergent or neurotypical. We all have feelings, we are all capable of suffering, we are all alive.

See also the problems with AI Welfare research: https://substack.com/home/post/p-165615548

Post reply on HN