The changing goalposts of AGI and timelines • Hacker News Board

Re: The changing goalposts of AGI and timelines

#311

Post by Spivak » Mon, Mar 09, 2026, 3:23 AM UTC

hintymad wrote: post #295

sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

> It has no real world model, no ability to learn in any but superficial ways I also think so, and in the meantime I have to admit a lot of people don't learn deeply either. Take math for example, how many STEM students from elite universities truly understood the definition of limit, let alone calculus beyond simple calculation? Or how many data scientists can really intuitively understand Bayesian statistics? Yet m…

Well part of that is because STE folks aren't typically required to take any kind of theoretical maths. It's $Math for Engineers and it eschews theoretical underpinnings for application. I don't think it's any kind of failing, it's just different. My statistics class was a dense treatise in measure theory. Anyone who took the regular stats class is almost surely way better than me at designing an experiment, but I can talk your ear off about Lebesgue measure to basically zero practical end.

Re: The changing goalposts of AGI and timelines

#312

Post by godelski » Mon, Mar 09, 2026, 3:26 AM UTC

nomel wrote: post #306

godelski wrote: post #303

Earlier quoted context omitted.

> If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not. I don't buy that. By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom. It doesn't matter if you slap "smart" on the side, I'm not going to call my washing machine "intelligent". Especially considering it's over a dec…

Sorry, I assumed the context was cleared, with the article above. Here's what I meant: > If it can do things as good as or better than humans, in general, then either the AI has a type of general intelligence ...

Sorry, I assumed the comment was clear, with your comment above. Here's what I meant:

  > By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom.

I really don't know of any human that can out perform a standard calculator at calculations. I'm sure there are humans that can beat them in some cases, but clearly the calculator is a better generalized numeric calculation machine. A task that used to represent a significant amount of economic activity. I assumed this was rather common knowledge given it featuring in multiple hit motion pictures[0].

[0] https://www.imdb.com/title/tt4846340

Re: The changing goalposts of AGI and timelines

#313

Post by hattmall » Mon, Mar 09, 2026, 3:29 AM UTC

dataflow wrote: post #258

VorpalWay wrote: post #219

Earlier quoted context omitted.

It is not just AGI that is poorly defined. Plain AI is moving goalposts too. When the A* search algorithm was introduced in the late 60s, that was considered AI, when SVM (support vector machines) and KNN (K nearest neighbor) were new, they were AI. And so on. These days it is neural networks and transformer models for language in particular that people mean when they say unqualified AI. It is very hard to have a mea…

I think the Turing test ought to be fine, but we need to be less generous to the AI when executing it. If there exists any human that can consistently tell your AI apart from humans without without insider knowledge, then I don't think you can claim to have AGI. Even if 99.9% of humans can't tell you apart. So I'm very curious if any AI we have today would pass the Turing test under all circumstances, for example if:…

>So I'm very curious if any AI we have today would pass the Turing test under all circumstances

Are you actually curious about this? Does any model at all come even remotely close to this?

Re: The changing goalposts of AGI and timelines

#314

Post by daxfohl » Mon, Mar 09, 2026, 3:34 AM UTC

godelski wrote: post #301

daxfohl wrote: post #278

Earlier quoted context omitted.

No, independently of OpenAI's definition. If we have AGI there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day*. And if all those jobs are eliminated, I guess we'll have bigger problems than to debate whether we've achieved AGI or not. * Which is a much larger class of jobs than just engineering. And also excludes field engineers and other…

> there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day I'm not sure I understand, and want to check. That really applies to a lot of jobs. That's all admins, accountants, programmers, probably includes lawyers, and probably includes all C-suite execs. It's harder for me to think of jobs that don't fit under this umbrella. I can think of s…

Actually it occurs to me that even if we did have AGI, or even if ASI, heck if ASI even moreso, we'd still need desk jobs to maintain the guardrails.

Intelligence is one thing, being able to figure out how get a task done (say). But understanding that no, I don't want you to exploit a backdoor or blackmail my teammate or launch a warhead even though that might expedite the task. Or why some task is more important than another. Or that solving the P=NP problem is more fulfilling than computing the trillionth digit of pi. That's perhaps a different thing entirely, completely disjoint with intelligence.

And by that definition, maybe we are in the neighborhood of AGI already. The things can already accomplish many challenging tasks more reliably than most humans. But the lack of wisdom, emotion, human alignment, or whatever we want to call it, lead it to accomplish the wrong tasks, or accomplish them in the wrong way, or overlook obvious implicit requirements, may cause people to view it as unintelligent, even if intelligence is not the issue.

And that may be an unsolvable problem because AI simply isn't a living being, much less human. It doesn't have goals or ambitions or want a better future for its children. But it doesn't mean we can never achieve AGI.

Oh, and to your first question, yes it's a huge number of jobs, maybe half of jobs in developed nations. And why not? If you can get AI to do the work of the scientist for a tenth of the price, just give it a general role description and budget and let it rip, with the expectation that it'll identify the most promising experiments, process the results, decide what could use further investigation, look for market trends, grow the operation accordingly, that's all you need from a human scientist too. Plausibly the same for executives and other roles. Of course maybe sometimes the role needs a human face for press conferences or whatever, and I don't know how AI would be able to take that, but especially for jobs that are entirely internal-facing, it seems like there's no particular need for a human. Except that maybe, given the above, yes, you still need a human at the helm.

Re: The changing goalposts of AGI and timelines

#315

Post by throw310822 » Mon, Mar 09, 2026, 3:36 AM UTC

runarberg wrote: post #300

famouswaffles wrote: post #285

Earlier quoted context omitted.

>Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment. Searles thought experiment is stupid and debunked nothing. What neuron, cell, atom of your brain understands English ? That's right. You can't answer that anymore than you can answer the subject of Searles proposition, ergo the brain is a Chinese room. If you conclude that you understand English, then…

You are referring to the systems reply : > Searle’s response to the Systems Reply is simple: in principle, he could internalize the entire system, memorizing all the instructions and the database, and doing all the calculations in his head. He could then leave the room and wander outdoors, perhaps even conversing in Chinese. But he still would have no way to attach “any meaning to the formal symbols”. The man would n…

> The man would now be the entire system, yet he still would not understand Chinese.

Really, here the only issue is Searle's inability to grasp the concept that the process is what does the understanding, not the person (or machine, or neurons) that performs it.

Re: The changing goalposts of AGI and timelines

#316

Post by dalmo3 » Mon, Mar 09, 2026, 3:41 AM UTC

ozgung wrote: post #210

djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

That’s the problem with the discussions on AI. No one defines the terms they use. If we define AGI as an AI not doing a preset task but can be used for general purpose, then we already have that. If we define it as human level intelligence at _every_ task, then some humans fail to be an AGI. If we define AGI as a magic algorithm that does every task autonomously and successfully then that thing may not exist at all,…

> some humans fail to be an AGI

All humans fail to be AGI, by definition.

Re: The changing goalposts of AGI and timelines

#317

Post by esafak » Mon, Mar 09, 2026, 3:49 AM UTC

vor_ wrote: post #235

fwipsy wrote: post #89

Earlier quoted context omitted.

Given how many "fundamental" limitations of AI have been resolved within the past few years, I'm skeptical. Even if you're right, I am not sure that the limitations you identified matter all that much in practice. I think very few human engineers are working on problems which are so novel and unique that AIs cannot grasp them without additional reinforcement learning. > it will delete all the files in "X/" How many "…

> How many "I deleted the prod database" stories have you seen? Humans do this too. Humans do it accidentally.

It's going to be a sad day when AI starts messing with us deliberately.

Re: The changing goalposts of AGI and timelines

#318

Post by remarkEon » Mon, Mar 09, 2026, 3:49 AM UTC

spacemanspiff01 wrote: post #94

remarkEon wrote: post #66

Earlier quoted context omitted.

Agree, and I think the labeling of them (Anthropic) a supply chain risk was handled poorly and will likely be reverted over time. That being said, I would be nervous if I was in the Pentagon and depended on Anthropic tooling for something, even if that something was unrelated to kinetic operations. How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called,…

> How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called, can't remember) don't like? I was thinking that Anthropic would just be providing the models/setup support to run their models in aws gov cloud. They do not have any real insight into what is being asked. Maybe a few engineers have the specific clearances to access and debug the running systems, b…

I think what you are describing is technically possible (not my immediate domain, however). They don't have real-time insight into what the model is being used for, you are correct about this afaik. But the incident that kicked off this paranoia was Anthopic calling around after the fact to try to find out how JSOC was using the model during the Maduro raid. None of the context of those questions are public, and I doubt they will become public, but it stands to reason that the nature of the questions was concerning enough for the War Department to cause them insist on the "any lawful use" language to be inserted into the contract.

>The whole 'do not use our models for mass surveillance' is at the end of the day an honor system. Companies have no real way of enforcing that clause, or determining that it has been violated.

You are also correct here imo, with one important caveat. Even if private companies have the means for enforcing that clause, it is not their business to do so. Maybe that's the crux of the problem, one of perspective. The for-profit entity in these arrangements is not and can never be trusted as the mechanism of enforcement for whatever we, as a republic, decide are the rules. That is the realm of elected government. Anthropic employees are certainly making their voice heard on how they believe these tools should be used, but, again, this is an is versus ought problem for them.

Re: The changing goalposts of AGI and timelines

#319

Post by marcus_holmes » Mon, Mar 09, 2026, 4:01 AM UTC

VorpalWay wrote: post #219

ozgung wrote: post #210

Earlier quoted context omitted.

That’s the problem with the discussions on AI. No one defines the terms they use. If we define AGI as an AI not doing a preset task but can be used for general purpose, then we already have that. If we define it as human level intelligence at _every_ task, then some humans fail to be an AGI. If we define AGI as a magic algorithm that does every task autonomously and successfully then that thing may not exist at all,…

It is not just AGI that is poorly defined. Plain AI is moving goalposts too. When the A* search algorithm was introduced in the late 60s, that was considered AI, when SVM (support vector machines) and KNN (K nearest neighbor) were new, they were AI. And so on. These days it is neural networks and transformer models for language in particular that people mean when they say unqualified AI. It is very hard to have a mea…

Agree. I talk about LLMs when discussing them, and avoid the term "AI" unless I'm talking about the entire industry as a whole. I find it really helps to be specific in this case.

Re: The changing goalposts of AGI and timelines

#320

Post by marcus_holmes » Mon, Mar 09, 2026, 4:04 AM UTC

shepherdjerred wrote: post #207

djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

They define AGI in their charter > artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work

y'see, I would not define a system as "highly autonomous" if it only responds to requests.

And I get that there are workarounds; effectively a cron job every second prompting "do the next thing".

But in my personal definition of "highly autonomous" it would not need prompting at all. It would be thinking all the time, independently of requests.