Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

331–340 of 411 posts

Re: The changing goalposts of AGI and timelines

#331

Post by sulam »

stavros wrote: post #248
sulam wrote: post #237

Earlier quoted context omitted.

Fair, I should define what I mean by under the hood. By “under the hood” I mean that models are still just being fed a stream of text (or other tokens in the case of video and audio models), being asked to predict the next token, and then doing that again. There is no technique that anyone has discovered that is different than that, at least not that is in production. If you think there is, and people are just keepin…

Everybody says "but they just predict tokens" as if that's not just "I hope you won't think too much about this" sleight of hand. Why does predicting the next token mean that they aren't AGI? Please clarify the exact logical steps there, because I make a similar argument that human brains are merely electrical signals propagating, and not real intelligence, but I never really seem to convince people.

Because there are some really fundamental things they cannot do with next token prediction. For instance, their memory is akin to someone who reads the phone book and memorizes the entire thing, but can't tell you what a phone number is for. Moreover, they can mimic semantic knowledge, because they have been trained on that knowledge, but take them out of their training distribution and they get into a "creative story-telling" mode very quickly. They can quote me all the rules of chess, but when it comes to actually making a chess move they break those rules with abandon simply because they didn't actually understand the rules. Chess is instructive in another way, too, in that you can get them to play a pretty solid opening game, maybe 10, 15 moves in, but then they start forgetting pieces, creating board positions that are impossible to reach, etc. They have memorized the forms of a board, know the names of the pieces, but they have no true understanding of what a chess game is. Coding is similar, they're fine when you give them Python or Bash shell scripts to write, they've been heavily trained on those, but ask them to deal with a system that has a non-standard stack and they will go haywire if you let their context get even medium sized. Something else they lack is any kind of learning efficiency as you or I would understand the concept. By this I mean the entire Internet is not sufficient to train today's models, the labs have to synthesize new data for models to train on to get sufficient coverage of a given area they want the model to be knowledgeable about. Continuous learning is a well-known issue as well, they simply don't do it. The labs have created memory, which is just more context engineering, but it's not the same as updating as you interact with them. I could go on.

At the end of the day next token prediction is a sleight of hand. It produces amazingly powerful affects, I agree. You can turn this one magic trick into the illusion of reasoning, but what it's doing is more of a "one thing after another" style story-telling that is fine for a lot of things, but doesn't get to the heart of what intelligence means. If you want to call them intelligent because they can do this stuff, fine, but it's an alien kind of intelligence that is incredibly limited. A dog or a cat actually demonstrate more ability to learn, to contextualize, and to make meaning.

Re: The changing goalposts of AGI and timelines

#332

Post by debugnik »

tokai wrote: post #150
logicchains wrote: post #145

Earlier quoted context omitted.

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.

Turing test is generally misunderstood, much like Schrodinger's cat, it has devolved in to a pop cultural meme. The test is to evaluate if a machine can think . Not if it is intelligent, not if it is human-like. Its dismissed as a useful by most experts in philosophy of mind, AI, language, etc.. Thinking cool and all but not that extraordinary. Even plants does it.

> The test is to evaluate if a machine can think.

The test is to showcase that the question of whether machines can think is meaningless. The point of Turing's thesis is that passing his test just proves the machine has the capability to pass such a test, which is actually meaningful.

Re: The changing goalposts of AGI and timelines

#333

Post by hunterpayne »

tokai wrote: post #150
logicchains wrote: post #145

Earlier quoted context omitted.

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.

Turing test is generally misunderstood, much like Schrodinger's cat, it has devolved in to a pop cultural meme. The test is to evaluate if a machine can think . Not if it is intelligent, not if it is human-like. Its dismissed as a useful by most experts in philosophy of mind, AI, language, etc.. Thinking cool and all but not that extraordinary. Even plants does it.

"Thinking cool and all but not that extraordinary. Even plants does it."

Are you involved in politics somehow?

Re: The changing goalposts of AGI and timelines

#334

Post by Dylan16807 »

gosub100 wrote: post #328
runarberg wrote: post #201

Earlier quoted context omitted.

I like the analogy with Schrödinger’s cat. Like Schrödinger’s cat it is actually not a good thought experiment. Both have been debunked. Schrödinger’s cat is applying quantum behavior (of a single interaction) to a macro system (with trillions of interactions). While the Turing test can be explained away with Searle’s Chinese room thought experiment. I would argue that Schrödinger’s cat has done more damage to the ge…

What do you mean by Schrodinger's cat experiment being "debunked"? The only way I can think to debunk it is to say there are ways to determine if the cat is alive such as heartbeat or temperature, which are impossible to isolate at a quantum level. I don't think anyone claimed the animal was in a superposition.

Debunked is a weird word since it was made to be absurd. But yes the issue is about whether the cat is in superposition, and real cats can't be.

Re: The changing goalposts of AGI and timelines

#335

Post by Dylan16807 »

runarberg wrote: post #287
djoldman wrote: post #253

Earlier quoted context omitted.

The Turing test and Searle's "rebuttal" are both pretty inconsequential. There's no real definition of "thinking," therefore neither proof/disprove or say much. Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ? I think most people would say that's an insufficient…

Searle’s rebuttal is actually excellent philosophy. But otherwise I agree. Searle was (just learned he passed away last year) a philosopher by trade, but Turing was a mathematician and Schrödinger was a theoretical physicist. So it is to be expected that a mathematician and a physicist might produce sub-par philosophy. Turing’s point in his 1950 paper was actually to provide a substitute to the question of whether ma…

Even if a Chinese room isn't a real boy, if it can do basically all text tasks at a human level I'm going to say it's capable of thinking. The issue of "understanding" can be left for another day (not that I think the Chinese room is very convincing on that front either).

I see no reason to disqualify p-zombies from being AGI.

Re: The changing goalposts of AGI and timelines

#336

Post by Wowfunhappy »

VorpalWay wrote: post #274
Wowfunhappy wrote: post #221

Earlier quoted context omitted.

I really wish I could wave a magic wand and make everyone stop using the term "AI". It means everything and nothing. Say "machine learning" if that's what you mean.

Machine learning: that definitely includes SVM and regression models. Oh and decision trees. Probably a few other things I'm not thinking of right now. Many people will unfortunately be thinking of just neural networks though. (By the way, if something like a regression model or decision tree can solve your problem, you should prefer those. Much cheaper to train and to run inference with those than with neural networ…

Wait, a decision tree is machine learning?

Re: The changing goalposts of AGI and timelines

#337

Post by DiscourseFan »

wise_blood wrote: post #330
djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

the ARC definition is the one I like the best, something like: "it is AGI when we can no longer come up with tasks easy for humans to solve but hard for computers"

We are very very far from that point

Re: The changing goalposts of AGI and timelines

#338

Post by stavros »

sulam wrote: post #331
stavros wrote: post #248

Earlier quoted context omitted.

Everybody says "but they just predict tokens" as if that's not just "I hope you won't think too much about this" sleight of hand. Why does predicting the next token mean that they aren't AGI? Please clarify the exact logical steps there, because I make a similar argument that human brains are merely electrical signals propagating, and not real intelligence, but I never really seem to convince people.

Because there are some really fundamental things they cannot do with next token prediction. For instance, their memory is akin to someone who reads the phone book and memorizes the entire thing, but can't tell you what a phone number is for. Moreover, they can mimic semantic knowledge, because they have been trained on that knowledge, but take them out of their training distribution and they get into a "creative stor…

None of this is a logical certainty of "X, therefore Y", it's just opinions. You can trivially add memory to a model by continuing to train it, we just don't do it because it's expensive, not because it can't be done.

Also, the phone book example is off the mark, because if I take a human who's never seen a phone and ask them to memorise the phone book, they would (or not), while not knowing what a phone number was for. Did you expect that a human would just come up on knowledge about phones entirely on their own, from nothing?

Re: The changing goalposts of AGI and timelines

#339

Post by orbital-decay »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

>You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do

I disagree that this prerequisite is more necessary than e.g. having legs to move over the ground. But besides that, current LLMs are literally a result of the continuous cycle of learning and deep memory. It's pretty crude compared to what evolution and human process had to do, but that's precisely how the iterative model development cycle with the hierarchical bootstrap looks like. It's not fully autonomous though (engineer-driven/humans in the loop). Moreover, the distillation process you describe is precisely what "learning" is.

Re: The changing goalposts of AGI and timelines

#340

Post by mirekrusin »

0xbadcafebee wrote: post #229
mirekrusin wrote: post #218

Earlier quoted context omitted.

I think you're somehow right and wrong at the same. All those "it's like ..." are faulty – "post-it notes" are not 3k pages of text that can be recalled instantly in one go, copied in fraction of a second to branch off, quickly rewritten, put into hierarchy describing virtually infinite amount of information (outside of 3k pages of text limit), generated on the fly in minutes on any topic pulling all information avai…

> you can already stuff-in more information than human can remember during whole lifetime The human eye processes between 100GB and 800GB of data per day. We then continuously learn and adapt from this firehose of information, using short-term and long-term memory, which is continuously retrained and weighted. This isn't "book knowledge", but the same capability is needed to continuously learn and reason on a human-e…

Again, yes and no.

Humans don't have monopoly on intelligence.

We don't need to mimick every aspect of humans to have intelligence or intelligence surpassing human abilities.

"General general-intelligence" doesn't exist in nature, it never did.

Humans can't echolocate, can't do fast mental arithmetic reliably, can't hold more than ~7 items in working memory, systematically fail at probabilistic reasoning and are notoriously bad at long term planning under uncertainty etc.

Human intelligence is _specialized_ (for social coordination, language, and tool use in a roughly savanna like environment).

We call it "general (enough)" because it's the only intelligence we have to compare against — it's a sample size of one, and we wrote down this definition.

The AGI goalposts keep moving but that's argument supporting what I'm saying not the other way around.

When machines beat us at chess, we said "that's just search".

When AlphaFold solved protein folding, we said "that's just pattern matching".

When models write better code than most engineers, manage complex information, and orchestrate multi-step agentic workflows — we say "but can it really understand"?

The question isn't whether AI mimics human cognition/works at low level the same way.

It's whether it can do things that do matter to us.

Programming, information synthesis and self directed task orchestration capabilities that exploded in last weeks/months aren't narrow tasks and they do compound.

Systems that now can coherently, recursively search, write, run, evaluate, revise etc. while keeping in memory equivalent 3k pages of text etc. are simply better than humans, now, today, I see it myself, you can hear people saying it.

Following weeks and months will be flooded with more and more reports – it takes a bit of time to set everything up and the tooling is still a bit rough on the edges.

But it's here and it's general enough.

Post reply on HN