Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I don't think AGI is imminent

dlants.me

Post reply on HN
Print view View original

71–80 of 313 posts

Re: Why I don't think AGI is imminent

#71

Post by dimitri-vs »

pixl97 wrote: post #44
hi_hi wrote: post #11

How will we know if its AGI/Not AGI? (I don't think a simple app is gonna cut it here haha) What is the benchmark now that the Turing test has been blown out of the water?

There is a different way I look at this. Humans will never accept we created AI, they'll go so far as to say we were not intelligent in the first place. That is the true power of the AI effect.

And yet another way to look at it is maybe current LLM agents are AGI, but it turns out that AGI in this form is actually not that useful because of its many limitations and solving those limitations will be a slow and gradual process.

Re: Why I don't think AGI is imminent

#72

Post by dimitri-vs »

mikewarot wrote: post #2

I think that AGI has already happened, but it's not well understood, nor well distributed yet. OpenClaw, et al, are one thing that got me nudged a little bit, but it was Sammy Jankis[1,2] that pushed me over the edge, with force. It's janky as all get out, but it'll learn to build it's own memory system on top of an LLM which definitely forgets. [1] https://sammyjankis.com/ [2] https://news.ycombinator.com/item?id=47…

I really don't see why AGI can't be a spectrum and we just have very weak AGI and going from weak to strong will take many years, if it ever happens.

Re: Why I don't think AGI is imminent

#73

Post by Lerc »

hunterpayne wrote: post #66
Lerc wrote: post #34

I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before. It seems like a prediction like "Bob won't become a formula one driver in a minivan". It's true, but not very interesting. I…

Then you don't understand Machine Learning in any real way. Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size. Just making the model bigger doesn't work because of something called the curse of dimensionality. This is something we have discovered about every single problem and type of learning algorithm used in ML. For LLMs, we probably moved past the ideal…

Um, what? Are you interpreting scaling to mean adding parameters and nothing else?

I'm not entirely sure where you get your confidence that we've past the ideal model size from, but at least that's a clear prediction so you should be able to tell if and when you are proven wrong.

Just for the record, do you care to put an actual number on something we won't go past?

[edit] Vibe check on user comes out as

    Contrarian 45%
    Pedantic 35%
    Skeptical 15%
    Direct  5%
That's got to be some sort of record.

Re: Why I don't think AGI is imminent

#74

Post by andsoitis »

alex43578 wrote: post #35
andsoitis wrote: post #30

Earlier quoted context omitted.

> We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks. Humans discovered or invented all of those.

Only in small ways and very recently, evolutionarily speaking, were those things rewarded by natural selection (and even that has stopped nowadays).

I'm not sure that's a good way to think about it.

Evolution transcends hard lines in the temporal sand that "separate species".

It also took billions of years of evolution to get to humans. so, humans, on the grander scale of life, is also just a very recent development.

Re: Why I don't think AGI is imminent

#75

Post by card_zero »

Animats wrote: post #31

Now that understanding video and projecting what happens next indicates we're getting past the LLM problem of lacking a world model. That's encouraging. There's more than one way to do intelligence. Basic intelligence has evolved independently three times that we know of - mammals, corvids, and octopuses. All three show at least ape-level intelligence, but the species split before intelligence developed, and the brai…

[flagged]

Re: Why I don't think AGI is imminent

#76

Post by dimitri-vs »

simbleau wrote: post #8

AGI is a messy term, so to be concise, we have the models that can do work. What we lack is orchestration, management, and workflows to use models effectively. Give it 5 years and those will be built and they could be built using the models we have today (Opus 4.6 at the time of this message).

Manual orchestration is a brittle crutch IMO - you don't get to the moon by using longer and longer ladders. A powerful model in theory should be able to self orchestrate with basic tools and environment. The thing is that it also might be as expensive as a human to run - from a tokens AND liability perspective.

Re: Why I don't think AGI is imminent

#77

Post by hi_hi »

esafak wrote: post #68
jobs_throwaway wrote: post #16

Earlier quoted context omitted.

Supranormal GDP growth is my bar. When its actually able to get around bottlenecks and produce value on a societal level

An agent need not have wants, so why would it try to increase its efficiency to obtain things?

I don't think that was the intent of the comment, more that true AGI should be so useful and transformative that it unlocks enough value and efficiencies to boost GDP. Much like the Industrial Revolution or harnessing electricity, instead of a fancy chatbot.

Re: Why I don't think AGI is imminent

#78

Post by esafak »

hi_hi wrote: post #77
esafak wrote: post #68

Earlier quoted context omitted.

An agent need not have wants, so why would it try to increase its efficiency to obtain things?

I don't think that was the intent of the comment, more that true AGI should be so useful and transformative that it unlocks enough value and efficiencies to boost GDP. Much like the Industrial Revolution or harnessing electricity, instead of a fancy chatbot.

Increased productivity is not equivalent to intelligence.

Re: Why I don't think AGI is imminent

#79

Post by dimitri-vs »

ryanSrich wrote: post #5

AGI is here. 90%+ of white collar work _can_ be done by an LLM. We are simply missing a tested orchestration layer. Speaking broadly about knowledge work here, there is almost nothing that a human is better at than Opus 4.6. Especially if you're a typical office worker whose job is done primarily on a computer, if that's all AGI is, then yeah, it's here.

API Opus 4.6 will tell you it's still 2025, admit it's wrong then revert back to being convinced it's 2025 as it nears it's context limit.

I'll go so far as to say LLM agents are AGI-lite but saying we "just need the orchestration layer" is like saying ok we have a couple neurons, now we just need the rest of the human.

Re: Why I don't think AGI is imminent

#80

Post by anonymid »

NiloCK wrote: post #61

> The transformer architectures powering current LLMs are strictly feed-forward. This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully. Heck, it was untrue before that as well, any time an LLM respon…

Thanks for reading, and I really appreciate your comments!

> who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully

Ah, this is a great point, and not something that I considered. I agree that the token feedback does change the complexity, and it seems that there's even a paper by the same authors about this very thing! https://arxiv.org/abs/2310.07923

I'll have to think on how that changes things. I think it does take the wind out of the architecture argument as it's currently stated, or at least makes it a lot more challenging. I'll consider myself a victim of media hype on this, as I was pretty sold on this line of argument after reading this article https://www.wired.com/story/ai-agents-math-doesnt-add-up/ and the paper https://arxiv.org/pdf/2507.07505 ... who brush this off with:

>Can the additional think tokens provide the necessary complexity to correctly solve a problem of higher complexity? We don't believe so, for two fundamental reasons: one that the base operation in these reasoning LLMs still carries the complexity discussed above, and the computation needed to correctly carry out that very step can be one of a higher complexity (ref our examples above), and secondly, the token budget for reasoning steps is far smaller than what would be necessary to carry out many complex tasks.

In hindsight, this doesn't really address the challenge.

My immediate next thought is - even solutions up to P can be represented within the model / CoT, do we actually feel like we are moving towards generalized solutions, or that the solution space is navigable through reinforcement learning? I'm genuinely not sure about where I stand on this.

> I don't have an opinion on this, but I'd like to hear more about this take.

I'll think about it and write some more on this.

Post reply on HN