AGI is an engineering problem, not a model training problem • Hacker News Board

Re: AGI is an engineering problem, not a model training problem

#231

Post by ivraatiems » Sun, Aug 24, 2025, 4:48 AM UTC

I think I am coming to agree with the opinions of the author, at least as far as LLMs not being the key to AGI on their own. The sheer impressiveness of what they can do, and the fact they can do it with natural language, made it feel like we were incredibly close for a time. As we adjust to the technology, it starts to feel like we're further away again.

But I still see all the same debates around AGI - how do we define it? what components would it require? could we get there by scaling or do we have to do more? and so on.

I don't see anyone addressing the most truly fundamental question: Why would we want AGI? What need can it fulfill that humans, as generally intelligent creatures, do not already fulfill? And is that moral, or not? Is creating something like this moral?

We are so far down the "asking if we could but not if we should" railroad that it's dazzling to me, and I think we ought to pull back.

Re: AGI is an engineering problem, not a model training problem

#232

Post by buu700 » Sun, Aug 24, 2025, 4:50 AM UTC

kelnos wrote: post #111

xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

> Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? That wouldn't have occurred to me, to be honest. To me, AGI is Data from Star Trek. Or at the very least, Arnold Schwarzenegger's character from The Terminator. I'm not sure that I'd make sentience a hard requirement for AGI, but I think my general mental fantasy of AGI even includes sentience. Claude Co…

I would categorize sentient AGI as artificial consciousness[1], but I don't see an obvious reason AGI inherently must be conscious or sentient. (In terms of near-term economic value, non-sentient AGI seems like a more useful invention.)

For me, AGI is an AI that I could assign an arbitrarily complex project, and given sufficient compute and permissions, it would succeed at the task as reliably as a competent C-suite human executive. For example, it could accept and execute on instructions to acquire real estate that matches certain requirements, request approvals from the purchasing and legal departments as required, handle government communication and filings as required, construct a widget factory on the property using a fleet of robots, and operate the factory on an ongoing basis while ensuring reliable widget deliveries to distribution partners. Current agentic coding certainly feels like magic, but it's still not that.

1: https://en.wikipedia.org/wiki/Artificial_consciousness

Re: AGI is an engineering problem, not a model training problem

#233

Post by germandiago » Sun, Aug 24, 2025, 4:55 AM UTC

jibal wrote: post #160

germandiago wrote: post #152

AGI cannot possibly exist.

Proof?

Well, you should show the proof that it is possible also, So it would be a draw.

I really think it is not possible to get that from a machine. You can improve and do much fancier than now.

But AGI would be something entirely different. It is a system that can do everything better than a human including creativity, which I believe it to be exclusively human as of now.

It can combine, simulate and reason. But think out of the box? I doubt so. It is different to being able to derive ideas from which human would create. For that it can be useful. But that would not be AGI.

Re: AGI is an engineering problem, not a model training problem

#234

Post by Arkid » Sun, Aug 24, 2025, 4:55 AM UTC

It is an approach problem. You can engineer it as much as you want but the current architectures wont get us to AGI. I have a feeling that we will end up over engineering on an approach which doesn't get us anywhere. We will make it work via guardrails and memory and context and what not but it wont get us where we want to be.

Re: AGI is an engineering problem, not a model training problem

#235

Post by joe_the_user » Sun, Aug 24, 2025, 4:58 AM UTC

hnuser123456 wrote: post #47

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Well,

Original 80s AI was based on mathematical logic. And while that might not encompass all philosophy, it certainly was a product of philosophy broadly speaking - some analytical philosophers could endorse. But it definitely failed and failed because it could process uncertainty (imo). I think also if you closely, classical philosophy wasn't particularly amenable to uncertainty either.

If anything, I would say that AI has inherited its failure from philosophy's failure and we should look to alternative approaches (from Cybernetics to Bergson to whatever) for a basis for it.

Re: AGI is an engineering problem, not a model training problem

#236

Post by glitchc » Sun, Aug 24, 2025, 5:00 AM UTC

sixo wrote: post #11

glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.

That's covered in the biological construct part.

And no, we definitely do have quantum computers. They're just not practical yet.

Re: AGI is an engineering problem, not a model training problem

#237

Post by glitchc » Sun, Aug 24, 2025, 5:02 AM UTC

slashdave wrote: post #205

glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

> We don't know if AGI is even possible outside of a biological construct yet Of course it is. A brain is just a machine like any other.

Except we don't understand how the brain actually works and have yet to build a machine that behaves like it.

Re: AGI is an engineering problem, not a model training problem

#238

Post by fuckaj » Sun, Aug 24, 2025, 5:02 AM UTC

tomrod wrote: post #197

fuckaj wrote: post #110

Earlier quoted context omitted.

The physical universe has much higher throughput and lower latency than our computer emulating a digital world.

Wouldn't it be nice if LLMs emulated the real world! They predict next likely text token. That we can do so much with that is an absolute testament to the brilliance of researchers, engineers, and product builders. We are not yet creating a god in any sense.

I mean that the computing power available to evolution and biological processes for training is magnitudes higher than for an LLM.

Re: AGI is an engineering problem, not a model training problem

#239

Post by aussieguy1234 » Sun, Aug 24, 2025, 5:02 AM UTC

The memory part makes sense. If a human brains working memory were to be filled with even half the context that you see before LLMs start to fail, it too would loose focus. That's why the brain has short term working memory and long term memory for things not needed in the moment.

Re: AGI is an engineering problem, not a model training problem

#240

Post by DrewADesign » Sun, Aug 24, 2025, 5:10 AM UTC

glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

Sometimes I think we’re like cats that learned how to make mirrors without really understanding them, and are so close to making one good enough that the other cat becomes sentient.