Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

171–180 of 411 posts

Re: The changing goalposts of AGI and timelines

#171

Post by croes »

MichaelDickens wrote: post #21
croes wrote: post #5

> Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do > It can be debated whether arena.ai is a suitable metric for AGI, a strong case can probably be made for why it’s not. However, that’s irrelevant, as the spirit of the self-sacrifice clause is to avoid an arms race, and we are clearly in one. No, the spirit is clearly meant for near AGI and we aren’t near AGI

Altman has personally claimed that we are close to AGI. Therefore, according to him, OpenAI should invoke the self-sacrifice clause.

Of course he claims that, he seeks money from investors but the charter is likely be written by people who took it seriously

Re: The changing goalposts of AGI and timelines

#172

Post by stratos123 »

linkregister wrote: post #162
stratos123 wrote: post #144

Earlier quoted context omitted.

> AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. The statements of what "actual researchers" are you relying upon for your "next 30 years" estimate? How do you reconcile them with the sub-10- or even sub-5-years timelines of other AI researchers, like Daniel Kokota…

I think you are overindexing on the integer value given in the parent post, rather than seeing the essence that LLMs in their current form only excel on tasks they have been specifically trained for. Karpathy himself has publicly stated that AGI itself is only possible with a new paradigm (that his group is working toward). He claims RHLF and attention models are near the end of their logarithmic curve. The concept o…

Personally I'm not even sold on the current paradigm being too limited to produce AGI - there are still several OOMs worth of compute increase available, plus the algorithmic improvements have overall been accumulating faster than predicted.

But even assuming that a major breakthrough is required, it seems ludicrous to me to go from that to a timeline of a decade or more. This isn't like fusion power research, where you spend 10 years building a new installation only to find new problems. Software development is inherently faster, and AI research in particular has been moving extremely quickly in the past. (GPT-3 is only 6 years old.) I don't think a wall in AI progress, if one comes at all, will last more than a few years.

Re: The changing goalposts of AGI and timelines

#173

Post by orbital-decay »

logicchains wrote: post #145
djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.

The most pragmatic definition I know is OpenAI's own: "highly autonomous systems that outperform humans at most economically valuable work". Which is still something between skippetyboop and zingybang, as it leaves a ton of room for OAI to decide if that moment is reached, and also economically valuable work is a moving target.

Re: The changing goalposts of AGI and timelines

#174

Post by Balinares »

MadxX79 wrote: post #152
sulam wrote: post #131

Earlier quoted context omitted.

Sorry, but you're mistaking outputs with process. If you actually know what models are doing under the hood to product output that (admittedly) looks very convincing, you'll quickly realize that they are simply exceptionally good at statistically predicting the next token in a stream of tokens. The reason you are having to become an expert at context engineering, and the reason the labs still hire engineers, is becau…

I enjoyed playing mastermind with LLMs where they pick the code and I have to guess it. It's not aware that it doesn't know what the code is (it isn't in the context because it's supposed to be secret), but it just keeps giving clues. Initially it works, because most clues are possible in the beginning, but very quickly it starts to give inconsistent clues and eventually has to give up. At no point does it "realise"…

That is actually a genius and beautifully simple way to exhibit the difference between thought and the appearance of thought.

Re: The changing goalposts of AGI and timelines

#175

Post by MadxX79 »

stratos123 wrote: post #167
MadxX79 wrote: post #153

Earlier quoted context omitted.

I'm guessing they have a lot of shares in the AI companies they work(ed) for, and they would like to pump their value so they can buy an even nicer carribean island than they can already afford?

Kokotajlo in particular is notable for being the guy who quit OpenAI in 2024 in protest of their policy of requiring researchers to abide by a non-disparagement agreement to retain their equity. In the end OpenAI caved and changed their policy, but if he was lying all along to inflate the value of his shares, it would have been quite a 4d chess move of him to gamble the shares themselves on doing so.

Isn't it just that he left way before gpt-5, then? At that point a sufficiently naive person could have believed that scaling was going to lead to AGI, but that sort of optimism died after he was already an outsider.

Re: The changing goalposts of AGI and timelines

#176

Post by mlinhares »

BoxFour wrote: post #87
mlinhares wrote: post #81

Earlier quoted context omitted.

It’s not obvious that the government should have to power to overwrite this, the US constitution was written as a collection of negative rights exactly to rein in government dictatorial impulses. And now that we see the government blatantly disrespecting the constitution and the rule of law the civil community must react.

> It’s not obvious that the government should have to power to overwrite this The government shouldn’t be able to set the terms of its contracts with private companies and walk away if those terms aren’t acceptable? That seems like a stretch. The constitution is a wildly different premise from government contracting with private companies.

There was no contract, the government wanted to have a contract where they'd be able to use the tool to violate privacy rights of its citizens and issue kill orders without a human present and the company said no.

The government shouldn't be able to coerce a business to do whatever it wants.

Re: The changing goalposts of AGI and timelines

#177

Post by mattlondon »

kovek wrote: post #164
mattlondon wrote: post #84

Earlier quoted context omitted.

I disagree. There is some argument to be had that they're already generally intelligent. They're already certainly better than me in basically anything I can ask them to do. So that leads to the question of what qualifies as intelligent? And do we need sentience for intelligence? What about self-agency/-actuation? Is that needed for "generally intelligent"? I don't know. But I feel like we're not there yet, even for…

Models need pre-training and fine tuning. Humans can do online learning.

How much of our brain's innate wiring is "pre-training"? We're born pre-wired to breath, to swallow, to blink, sleep, cry etc. Fine tuned over many many many epochs and baked into our model weights/DNA.

Is a newborn baby without learnt-knowledge not an intelligent being to you?

Or is an empty vessel such as a newborn baby intelligent merely because it has the ability to learn?

It gets pretty philosophical pretty quick. This is why I don't think there'll be a "moment" when AGI happens - there is so many ways to interpret what constitutes intelligence.

But yeah I agree that until models can learn in real time then I think we're probably not there yet. As I said - 5 years give or take I reckon.

Re: The changing goalposts of AGI and timelines

#178

Post by edanm »

ufmace wrote: post #169
sebastos wrote: post #156

Earlier quoted context omitted.

Firstly, the models that pass the Math Olympiad aren’t the same models as the ones you’re saying “pass the Turing test”. Secondly, nothing actually passes the Turing test. They pass a vibes check of “hey that’s pretty good!” but if your life depended on it, you could easily find ways to sniff out an LLM agent. Thirdly, none of these models learn in real time, which is an obviously essential feature. We’ll know AGI wh…

> nothing actually passes the Turing test Says who? I had already found this study, published almost a year ago, saying that they do: https://arxiv.org/abs/2503.23674 There doesn't seem to be a super-rigorous definition of the Turing Test, but I don't think it's reasonable to require it to fool an expert whose life depends on the correct choice. It already seems to be decently able to fool a person of average intelli…

Turing gave a pretty rigorous definition of the Turing Test IMO. Well, as rigorous as something that is inherently "anecdotal" can be, which is part of the philosophical point of the Turing Test.

Re: The changing goalposts of AGI and timelines

#179

Post by politelemon »

djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

The enskibidification of AI

Re: The changing goalposts of AGI and timelines

#180

Post by [deleted] »

surgical_fire wrote: post #109
fwipsy wrote: post #89

Earlier quoted context omitted.

Given how many "fundamental" limitations of AI have been resolved within the past few years, I'm skeptical. Even if you're right, I am not sure that the limitations you identified matter all that much in practice. I think very few human engineers are working on problems which are so novel and unique that AIs cannot grasp them without additional reinforcement learning. > it will delete all the files in "X/" How many "…

> Given how many "fundamental" limitations of AI have been resolved within the past few years Eh? Which limitations were solved?

[deleted]
Post reply on HN