The changing goalposts of AGI and timelines • Hacker News Board

Re: The changing goalposts of AGI and timelines

#181

Post by Copyrightest » Sun, Mar 08, 2026, 8:40 PM UTC

ufmace wrote: post #169

sebastos wrote: post #156

Earlier quoted context omitted.

Firstly, the models that pass the Math Olympiad aren’t the same models as the ones you’re saying “pass the Turing test”. Secondly, nothing actually passes the Turing test. They pass a vibes check of “hey that’s pretty good!” but if your life depended on it, you could easily find ways to sniff out an LLM agent. Thirdly, none of these models learn in real time, which is an obviously essential feature. We’ll know AGI wh…

> nothing actually passes the Turing test Says who? I had already found this study, published almost a year ago, saying that they do: https://arxiv.org/abs/2503.23674 There doesn't seem to be a super-rigorous definition of the Turing Test, but I don't think it's reasonable to require it to fool an expert whose life depends on the correct choice. It already seems to be decently able to fool a person of average intelli…

[dead]

Re: The changing goalposts of AGI and timelines

#182

Post by paulryanrogers » Sun, Mar 08, 2026, 8:42 PM UTC

stratos123 wrote: post #172

linkregister wrote: post #162

Earlier quoted context omitted.

I think you are overindexing on the integer value given in the parent post, rather than seeing the essence that LLMs in their current form only excel on tasks they have been specifically trained for. Karpathy himself has publicly stated that AGI itself is only possible with a new paradigm (that his group is working toward). He claims RHLF and attention models are near the end of their logarithmic curve. The concept o…

Personally I'm not even sold on the current paradigm being too limited to produce AGI - there are still several OOMs worth of compute increase available, plus the algorithmic improvements have overall been accumulating faster than predicted. But even assuming that a major breakthrough is required, it seems ludicrous to me to go from that to a timeline of a decade or more. This isn't like fusion power research, where…

> there are still several OOMs worth of compute increase available

Where and how? Aren't we reaching the physical limits of making transistors smaller?

Re: The changing goalposts of AGI and timelines

#183

Post by atomicnumber3 » Sun, Mar 08, 2026, 8:54 PM UTC

politelemon wrote: post #179

djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

The enskibidification of AI

Honestly, not enough of a joke.

I was thinking something similar - this isn't AI, and none of "those people" care if it is or isn't. They don't care philosophically, or even pragmatically.

They're selling a product. That product is the IDEA of replacement of the majority of human labor with what's basically slave labor but with substantially disregardable ethical quandaries.

It's honestly a genius product. I'm not surprised it's selling so well. I'm vaguely surprised so many people who don't stand to benefit in any way shape or form, or who will even potentially starve if it works out, are so keen on it. But there are always bootlickers.

The most unfortunate part is that when the party ends, it's none of "those people" who will suffer even in the slightest. I'm not even optimistic their egos will suffer, as Musk seems to show they are utterly immune even as their companies collapse under them.

Re: The changing goalposts of AGI and timelines

#184

Post by [deleted] » Sun, Mar 08, 2026, 9:05 PM UTC

mattlondon wrote: post #97

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

I don't think humans learn any differently than post it notes TBH We call them text books though!

[deleted]

Re: The changing goalposts of AGI and timelines

#185

Post by Jensson » Sun, Mar 08, 2026, 9:07 PM UTC

hirvi74 wrote: post #96

wongarsu wrote: post #52

Earlier quoted context omitted.

AGI is so nebulous we will never be able to tell if we hit it. We have hit human-level abilities in some narrow tasks, and are still leagues away in others. And humans have so vastly different skill-levels that we can't even agree what human-level really means. As bad as the economic definition of AGI in OpenAI's Microsoft deal is, at least it's measurable. Imho that's a big part of why people are shifting to ASI. No…

> AGI is so nebulous we will never be able to tell if we hit it. I completely agree. We can't even measure each other well, let alone machines.

It is very easy to tell if we still need humans in the loop. We still do so its not AGI.

Re: The changing goalposts of AGI and timelines

#186

Post by ACCount37 » Sun, Mar 08, 2026, 9:19 PM UTC

rishabhaiover wrote: post #137

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

I don't agree. The recent emergent behavior displayed by LLMs and test-time scaling (10x YoY revenue for Anthropic) is worth some hype. Of course, you are correct that most people who rally behind AGI do not understand the fundamental limitations of next-token prediction.

The "fundamental limitations" being what exactly?

Re: The changing goalposts of AGI and timelines

#187

Post by ACCount37 » Sun, Mar 08, 2026, 9:22 PM UTC

sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

Given the mechanistic interpretability findings? I'm not sure how people still say shit like "no real world model" seriously.

Re: The changing goalposts of AGI and timelines

#188

Post by charcircuit » Sun, Mar 08, 2026, 9:28 PM UTC

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

LLMs are AGI because they offer intelligence on any subject.

>the same ability to perform tasks as a human

The first chess AIs lost to chess grandmasters. AI does not need to be better than humans to be considered AI.

>without a continuous cycle of learning and deep memory, which LLMs cannot do.

But harnesses like Claude Code can with how they can store and read files along with building tools to work with them.

>which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you

This don't matter. You could say a chess AI is a bunch of different people who work together to explore distant paths of the search space. The idea you can split things into steps does not disqualify it from being AI.

>But tell one "don't delete files in X/", and after a while, it will delete all the files in "X/"

Humans make mistakes and mess up things too. LLMs are better at needle in a haystack tests than humans.

>It also does fun stuff like follow arbitrary instructions from an attacker

A ton of people get phished or social engineered by attackers. This is the number 1 way people get hacked. Do not underestimate people's willingness to follow instructions from strangers.

Re: The changing goalposts of AGI and timelines

#189

Post by sulam » Sun, Mar 08, 2026, 9:29 PM UTC

ACCount37 wrote: post #187

sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

Given the mechanistic interpretability findings? I'm not sure how people still say shit like "no real world model" seriously.

They have a _text_ model. There is some correlation between the text model and the world, but it’s loose and only because there’s a lot of text about the world. And of course robotics researchers are having to build world models, but these are far from general. If they had a real world model, I could tell them I want to play a game of chess and they would be able to remember where the pieces are from move to move.

Re: The changing goalposts of AGI and timelines

#190

Post by coliveira » Sun, Mar 08, 2026, 9:31 PM UTC

bluefirebrand wrote: post #168

CamperBob2 wrote: post #105

Earlier quoted context omitted.

For example, most companies would swear they were decided on cutting carbon emissions, just to forget everything when it comes to build data centers. This doesn't seem contradictory if you consider that success at AGI will solve the problem of carbon emissions, one way or another. If one data center ultimately replaces a whole medium-sized city of commuters...

> If one data center ultimately replaces a whole medium-sized city of commuters... Then we find out how long it takes for a medium sized city of commuters to start killing each other, elites and burning down data centers. Once they're hungry enough it'll happen for sure

Thinking about that, a world dominated by data centers will be relatively easy do disrupt, someone just needs to destroy a dozen or so datacenters to bring everything down.