Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

171–180 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#171

Post by danenania »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I’m not sure that self-updating weights is really analogous to “continuous learning” as humans do it. A memory data structure that the model can search efficiently might be a lot closer.

Self-updating weights could be more like epigenetics.

Re: AGI is an engineering problem, not a model training problem

#172

Post by walleeee »

Davidzheng wrote: post #133
walleeee wrote: post #88

Earlier quoted context omitted.

> It is vacuously true that a Turing machine can implement human intelligence The case of simulating all known physics is stronger so I'll consider that. But still it tells us nothing, as the Turing machine can't be built. It is a kind of tautology wherein computation is taken to "run" the universe via the formalism of quantum mechanics, which is taken to be a complete description of reality, permitting the assumptio…

QM is a testable hypothesis, so I don't think it's necessarily like an axiomatic assumption here. I'm not sure what you mean by "it tells us nothing, as ... can't be built". It tells us there's no theoretical constraint and only an engineering constraint to doing simulating the human brain (and all the tasks)

Sure, you can simulate a brain. If and when the simulation starts to talk you can even claim you understand how to build human intelligence in a limited sense. You don't know if it's a complete model of the organism until you understand the organism. Maybe you made a p zombie. Maybe it's conscious but lacks one very particular faculty that human beings have by way of some subtle phenomena you don't know about.

There is no way to distinguish between a faithfully reimplemented human being and a partial hackjob that happens to line up with your blind spots without ontological omniscience. Failing that, you just get to choose what you think is important and hope it's everything relevant to behaviors you care about.

Re: AGI is an engineering problem, not a model training problem

#173

Post by galangalalgol »

Uehreka wrote: post #116
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

Re: AGI is an engineering problem, not a model training problem

#174

Post by jibal »

signa11 wrote: post #83

won’t somebody please think about Mr. Godel, and the Incompleteness Theorem ?

They aren't relevant. Even if Penrose and Lucas were right (they aren't), a computational system can solve the vast majority of the problems we would want solved.

Re: AGI is an engineering problem, not a model training problem

#175

Post by mindcrime »

epiccoleman wrote: post #82
tshaddox wrote: post #50

Earlier quoted context omitted.

That sounds like you’re describing AGI as being impractical to implement in an electronic computer, not impossible in principle.

Yeah, I guess I'm not taking a stance on that above, just wondering where in that chain holds the most explanatory power for intelligence and/or consciousness. I don't think there's any real reason to think intelligence depends on "meat" as its substrate, so AGI seems in principle possible to me. Not that my opinion counts for much on this topic, since I don't really have any relevant education on the topic. But my h…

But my half baked instinct is that LLMs in and of themselves will never constitute true AGI.

I agree. But... LLM's are not the only game in town. They are just one approach to AI that is currently being pursued. The current dominant approach by investment dollars, attention, and hype, to be sure. But still far from the only thing around.

Re: AGI is an engineering problem, not a model training problem

#177

Post by dsign »

I have colleagues that want to plan each task of the software team for the next 12 months. They assume that such a thing is possible, or they want to do it anyway because management tells them to. The first would be an example of human fallibility, and the second would be an example of choosing the path of (perceived) least immediate self-harm after accounting for internal politics.

I doubt very much we will ever build a machine that has perfect knowledge of the future or that can solve each and every “hard” reasoning problem, or that can complete each narrow task in a way we humans like. In other words, it’s not simply a matter of beating benchmarks.

In my mind at least, AGI’s definition is simple: anything that can replace any human employee. That construct is not merely a knowledge and reasoning machine, but also something that has a stake on its own work and that can be inserted in a shared responsibility graph. It has to be able to tell that senior dev “I know planning all the tasks one year in advance is busy-work you don’t want to do, but if you don’t, management will terminate me. So, you better do it, or I’ll hack your email and show everybody your porn subscriptions.”

Re: AGI is an engineering problem, not a model training problem

#178

Post by conartist6 »

AGI has a hundreds-of-millions of years of evolution problem, anything humans have done so far utterly pales in comparison. The lowliest rat has more "general" intelligence than any AI we've ever made...

If we want to learn, look to nature, and it *has to be alive*.

Re: AGI is an engineering problem, not a model training problem

#179

Post by therobots927 »

nikkwong wrote: post #164
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

I found it strange that John Carmack and Ilya Sutskever both left prestigious positions within their companies to pursue AGI as if they had some proprietary insight that the rest of industry hadn't caught on to. To make as bold of a career move that publicly would mean you'd have to have some ultra serious conviction that everyone else was wrong or naive and you were right. That move seemed pompous to me at the time;…

The simple explanation is that they got high on their own supply. They deluded themselves into thinking an LLM was on the verge of consciousness.

Re: AGI is an engineering problem, not a model training problem

#180

Post by dataviz1000 »

zdragnar wrote: post #86
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Claude code is neither sentient nor sapient. I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is. The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.

Student: How do I know I exist?

Philosophy Professor: Who is asking?

Student: I am!

Post reply on HN