Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

181–190 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#181

Post by jasonwatkinspdx »

nikkwong wrote: post #164
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

I found it strange that John Carmack and Ilya Sutskever both left prestigious positions within their companies to pursue AGI as if they had some proprietary insight that the rest of industry hadn't caught on to. To make as bold of a career move that publicly would mean you'd have to have some ultra serious conviction that everyone else was wrong or naive and you were right. That move seemed pompous to me at the time;…

With Carmack it's consciously a dilliante project.

He's been effectively retired for quite some time. It's clear at some point he no longer found game and graphics engine internals motivation, possibly because the industry took the path he was advocating against back in the day.

For a while he was focused on Armadillo aerospace, and they got some cool stuff accomplished. That was also something of a knowing pet project, and when they couldn't pivot to anything that looked like commercial viability he just put it in hibernation.

Carmack may be confident (ne arrogant) enough to think he does have something unique to offer with AGI, but I don't think he's under any illusions it's anything but another pet project.

Re: AGI is an engineering problem, not a model training problem

#182

Post by manquer »

tshaddox wrote: post #21
glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”

Not necessarily , for a given definition of AGI you could have mathematical proof that it is incomputable similar to how Gödel incompleteness theorems work .

It need not even be incomputable, it could be NP hard and practically be incomputable, or it could be undecidable I.e. a version of the halting problem.

There are any number of ways our current models of mathematics or computation can in theory could be shown as not capable of expressing AGI without needing a fundamental change in physics

Re: AGI is an engineering problem, not a model training problem

#183

Post by joe_the_user »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

So the "Bitter Lesson" paper actually came up recently and I was surprised to discover that what it claimed was sensible and not at "all you need is data" or "data is inherently better"

The first line and the conclusion is: "The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." [1]

I don't necessary agree with it's examples or the direction it vaguely points at. But it's basic statement seems sound. And I would say that there's lot of opportunity for engineer, broadly speaking, in the process of creating "general methods that leverage computation" (IE, that scale). What the bitter lesson page was roughly/really about was earlier "AI" methods based on logic-programming and which including information on the problem domain in the code itself.

And finally, the "engineering" the paper talks about actually is pro-Bitter lesson as far as I can tell. It's taking data routing and architectural as "engineering" and here I agree this won't work - but for the opposite reason - specifically 'cause I don't just data routing/process will be enough.

[1]https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

Re: AGI is an engineering problem, not a model training problem

#184

Post by EMIRELADERO »

galangalalgol wrote: post #173
Uehreka wrote: post #116

Earlier quoted context omitted.

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

I don't see how your explanation leads to consciousness not being a thing. Consciousness is whatever process/mechanisms there are that as a whole produce our subjective experience and all its sensations, including but not limited to touch, vision, smell, taste, pain, etc.

Re: AGI is an engineering problem, not a model training problem

#185

Post by [deleted] »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

[deleted]

Re: AGI is an engineering problem, not a model training problem

#186

Post by nprateem »

therobots927 wrote: post #179
nikkwong wrote: post #164

Earlier quoted context omitted.

I found it strange that John Carmack and Ilya Sutskever both left prestigious positions within their companies to pursue AGI as if they had some proprietary insight that the rest of industry hadn't caught on to. To make as bold of a career move that publicly would mean you'd have to have some ultra serious conviction that everyone else was wrong or naive and you were right. That move seemed pompous to me at the time;…

The simple explanation is that they got high on their own supply. They deluded themselves into thinking an LLM was on the verge of consciousness.

The simpler answer is they could convince VCs to give them boat loads of cash by sounding like they can.

Re: AGI is an engineering problem, not a model training problem

#187

Post by BenFranklin100 »

This guys gets it wrong too. It’s not even an engineering problem. It’s much worse: it’s a scientific problem. We don’t yet understand how the human brain operates or what human intelligence really is. There’s nothing to engineer as the basic specifications are not yet available on what needs to be built.

Will AGI require ‘consciousness’, another poorly understood concept? How are mammalian brain even wired up? The most advanced model is the Allen Institute’s Mesoscale Connectivity Atlas which is at best a low resolution static roadmap, not a dynamic description of how a brain operates in real time. And it describes a mouse brain, not a human brain which is far, far more complex, both in terms of number of parts, and architecture.

People are just finally starting to acknowledge LLMs are dead ends. The effort expended on them over the last five years could well prove a costly diversion along the road to AGI, which is likely still decades in the future.

Re: AGI is an engineering problem, not a model training problem

#188

Post by joe_the_user »

justcallmejm wrote: post #159
andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

The missing science to engineer intelligence is composable program synthesis. Aloe ( https://aloe.inc ) recently released a GAIA score demonstrating how CPS dramatically outperforms other generalist agents (OpenAI's deep research, Manus, and Genspark) on tasks similar to those a knowledge worker would perform. I'd argue it's because intelligence has been treated as a ML/NN engineering problem that we've had the hyper…

CPS sounds interesting but your link goes to a teaser trailer and a waiting list. It's kind of hard to expect much from that.

Re: AGI is an engineering problem, not a model training problem

#189

Post by tomrod »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

Aye. Missing are self correction (world models/action and response observation), coherence over the long term, and self-scaling. The 3rd are what all the SV types are worried about, except maybe Yann LeCun who is worried about the first and second.

Hinton thinks the 3rd is inevitable/already here and humanity is doomed. It's an odd arena.

Re: AGI is an engineering problem, not a model training problem

#190

Post by crystal_revenge »

ants_everywhere wrote: post #90
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

> there is missing philosophy I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.

The magical thinking around LLMs is getting bizarre now.

LLMs are not “intelligent” in any meaningful biological sense.

Watch a spider modify its web to adapt to changing conditions and you’ll realize just how far we have to go.

LLMs sometimes echo our own reasoning back at us in a way that sounds intelligent and is often useful, but don’t mistake this for “intelligence”

Post reply on HN