Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

351–360 of 411 posts

Re: The changing goalposts of AGI and timelines

#351

Post by datsci_est_2015 »

chrysoprace wrote: post #250
djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

I've largely avoided using the term "AI" to refer to the current LLM and generative technology because it's loaded with too much ambiguity and glosses over the problems with those technologies in the context of conversations around it.

Same, but I have used the term “generative AI” to describe generative models. Never the naked “AI” though (except in conversations with friends where the difference is pedantic because they’re not subject matter experts).

Re: The changing goalposts of AGI and timelines

#352

Post by xmcqdpt2 »

daxfohl wrote: post #273
djoldman wrote: post #214

Earlier quoted context omitted.

That definition is as I said: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." "Highly autonomous systems" and "most economically valuable work" aren't precise enough to be useful. "Highly" implies that there is a continuum, so where does directed end and autonomy begin? "Most economically valuable work"... each word in that has wiggle ro…

I think you can say if human engineers still exist, it's hard to claim we have AGI. If human engineers have been entirely replaced, then it's hard to claim we don't have AGI.

By the definition above, it is possible to have AGI that is also much more expensive to run than human engineers.

Re: The changing goalposts of AGI and timelines

#353

Post by xmcqdpt2 »

shepherdjerred wrote: post #207
djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

They define AGI in their charter > artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work

is it most as an 50% of individual jobs? or able to produce 50% dollar for dollar?

what does "economically" means here? would it cover teaching? child care? healthcare? etc.

Re: The changing goalposts of AGI and timelines

#354

Post by datsci_est_2015 »

devonkelley wrote: post #264

[flagged]

“Judgment” is close to my mental model, but I prefer “liability”. All meatspace-based employment comes down to liability. A McDonald’s shift worker is liable for any mistakes made during their shift that are considered their responsibility. A SWE is liable for any mistakes made during their employment that are considered their responsibility. Accountant. Paralegal. CEO.

AI will need to be able to experience consequences as a result of liability, and care about those consequences, in order to replace true meatspace jobs. Otherwise they’re simply sophisticated systems.

If you’re a one man company, and you have a delivery AI that delivers widgets to Alice, but in the process that delivery AI kills Bob, you’re liable for murder.

Re: The changing goalposts of AGI and timelines

#355

Post by rishabhaiover »

ACCount37 wrote: post #281
rishabhaiover wrote: post #271

Earlier quoted context omitted.

I used to think it was the quadratic complexity of attention but I guess that's not a concern anymore as they've made more hardware aware kernels of attention? The other I remember is continual learning but that may be solved in near-term future. I am not completely confident about it.

Humans do have an upper limit on how much working memory they have. Which I see as the closest thing to the "O(N^2) attention curse" of LLMs. That doesn't stop an LLM from manipulating its context window to take full advantage of however much context capacity it has. Today's tools like file search and context compression are crude versions of that.

Human brain's prediction loop is bayesian in nature.

Re: The changing goalposts of AGI and timelines

#356

Post by rishabhaiover »

rishabhaiover wrote: post #355
ACCount37 wrote: post #281

Earlier quoted context omitted.

Humans do have an upper limit on how much working memory they have. Which I see as the closest thing to the "O(N^2) attention curse" of LLMs. That doesn't stop an LLM from manipulating its context window to take full advantage of however much context capacity it has. Today's tools like file search and context compression are crude versions of that.

Human brain's prediction loop is bayesian in nature.

Damn, the research moves fast. I was wrong again: https://arxiv.org/abs/2507.11768

Re: The changing goalposts of AGI and timelines

#357

Post by MadxX79 »

dwohnitmok wrote: post #260
MadxX79 wrote: post #175

Earlier quoted context omitted.

Isn't it just that he left way before gpt-5, then? At that point a sufficiently naive person could have believed that scaling was going to lead to AGI, but that sort of optimism died after he was already an outsider.

Kokotajlo still believes we get AGI in the next few years. These are his most updated numbers at the moment: https://www.aifuturesmodel.com/

I love the total lack of humility on that site. "What if the METR study turns out not to capture anything relevant? We just add a constant gap to be conservative!". But I guess these guys aren't really scientist, so it's probably a lot to ask that they relate critically to what they are doing and be honest about the limitations of their methods.

What if it turns out that the more you scale the more your LLM resembles a lobotomized human. It looks like it goes really well in the beginning, but you are just never going to get to Einstein. How does that affect everything?

What if it turned out that those AI companies were maybe having a whole bunch of humans solving the problems that are currently just below the 50% reliability threshold they set, and do fine tuning with those solutions. That will make their models perform better on the benchmark, but it's just training for the test... will the constant gap be a good approximation then?

Re: The changing goalposts of AGI and timelines

#359

Post by RugnirViking »

ACCount37 wrote: post #217
sulam wrote: post #189

Earlier quoted context omitted.

They have a _text_ model. There is some correlation between the text model and the world, but it’s loose and only because there’s a lot of text about the world. And of course robotics researchers are having to build world models, but these are far from general. If they had a real world model, I could tell them I want to play a game of chess and they would be able to remember where the pieces are from move to move.

What makes you think that text is inherently a worse reflection of the world than light is? All world models are lossy as fuck, by the way. I could give you a list of chess moves and force you to recover the complete board state from it, and you wouldn't fare that much better than an off the shelf LLM would. An LLM trained for it would kick ass though.

> I could give you a list of chess moves and force you to recover the complete board state from it, and you wouldn't fare that much better than an off the shelf LLM would

idk, I would expect anyone with an understanding of the rules of chess, and an understanding of whatever notation the moves are in, would be able to do it reasonably well? does that really sound so hard to you? people used to play correspondance chess. Heck I remember people doing it over email.

In comparison, current ai models start to completely lose the plot after 15 or so moves, pulling out third, fourth and fifth bishops, rooks etc from thin air, claiming checkmate erroneously etc, to the point its not possible to play a game with them in a coherent manner.

Post reply on HN