Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

291–300 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#291

Post by adastra22 »

xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

No you are not the only one. I am continuously mystified by the discussion surrounding this. Clause is absolutely and unquestionably an artificial general intelligence. But what people mean by “AGI” is a constantly shifting, never defined goalpost moving at sonic speed.

Re: AGI is an engineering problem, not a model training problem

#292

Post by adastra22 »

kelnos wrote: post #111
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

> Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? That wouldn't have occurred to me, to be honest. To me, AGI is Data from Star Trek. Or at the very least, Arnold Schwarzenegger's character from The Terminator. I'm not sure that I'd make sentience a hard requirement for AGI, but I think my general mental fantasy of AGI even includes sentience. Claude Co…

I would love for you to define AGI in such a way as for that to make sense.

I presuppose that you actually mean ASI as a starting point, and that is being charitable that it isn’t just pattern matching to questionable sci-fi.

Re: AGI is an engineering problem, not a model training problem

#293

Post by 9dev »

root_axis wrote: post #255
galangalalgol wrote: post #173

Earlier quoted context omitted.

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

What does it mean for consciousness to be an illusion? That "illusion" is the bedrock for our shared definition of reality.

You can never know whether anyone else is actually conscious, or just appearing to be. This shared definition of reality was always on shaky ground, given that we don’t even have the same sensory input, and "now" isn’t the same concept everywhere. You are a collection of processes that work together to keep you alive. Part of that is something that collects your history to form a distinctive narrative of yourself, and something that lives in the moment and handles immediate action. This latter part is solidly backed up by experiments; Say you feel pain that varies over time. If the pain level is an 8 for 14 consecutive minutes, and a 2 for 1 minute at the end, you’ll remember the whole session as level 4. In practical terms, this means a physician can make a procedure be perceived as less painful by causing you wholly unnecessary mild pain for a short duration after the actual work is done.

This also means that there’s at least two versions of you inside your mind; one that experiences, and one that remembers. There’s likely others, too.

Re: AGI is an engineering problem, not a model training problem

#294

Post by tmountain »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I’m no expert, but it seems like self updating weights requires a grounded understanding of the underlying subject matter, and this seems like a problem current LLM systems.

Re: AGI is an engineering problem, not a model training problem

#295

Post by 9dev »

nikkwong wrote: post #164
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

I found it strange that John Carmack and Ilya Sutskever both left prestigious positions within their companies to pursue AGI as if they had some proprietary insight that the rest of industry hadn't caught on to. To make as bold of a career move that publicly would mean you'd have to have some ultra serious conviction that everyone else was wrong or naive and you were right. That move seemed pompous to me at the time;…

Not sure about that. Think of Avi Loeb, for example, a brilliant astrophysicist and Harvard professor who recently became convinced that the interstellar objects traversing the solar system are actually alien probes scouting the solar system. He’s started a program called "Galileo" now to find the aliens and prepare people for the truth.

So I don’t think brilliance protects from derailing…

Re: AGI is an engineering problem, not a model training problem

#296

Post by JSR_FDED »

Jensson wrote: post #274
JSR_FDED wrote: post #213

Earlier quoted context omitted.

Interesting, I hadn’t thought about it that way. But can a thing on the other end of an API call ever truly have a “stake“?

> But can a thing on the other end of an API call ever truly have a “stake“? That is their goal function they are trained for, it is like dopamine and sex for humans they will do anything to get it.

Yes, but a having a stake also implies feeling the loss if it goes sideways…

Next you’re going to tell me that’s what loss functions are for :-)

Re: AGI is an engineering problem, not a model training problem

#297

Post by ClayShentrup »

Uehreka wrote: post #116
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

consciousness has to be fundamental.

Re: AGI is an engineering problem, not a model training problem

#298

Post by keepamovin »

cakealert wrote: post #194

There is a reason why LLM's are architected the way they are and why thinking is bolted on. The architecture has to allow for gradient descent to be a viable training strategy, this means no branching (routing is bolted on). And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't…

This is so interesting. It’s suggests that a kind of thought sensing brain scanning technology could be used as training data for the nonverbal thought layer.

I guess smart people in big companies already consider this and are currently working on technologies for products That will include some form of electromagnetic brain sensing - Provided conveniently as an interface - but also usefully a source of this data.

It also suggests to me that AI/AGI is far more susceptible to traditional disruption than the narratives of established incumbents suggest. You could have a Kickstarter like killer product, including such a headset that would provide the data to bootstrap that startup’s super AI.

Exciting times!

Re: AGI is an engineering problem, not a model training problem

#299

Post by Jensson »

exe34 wrote: post #275
frabcus wrote: post #202

Earlier quoted context omitted.

You've missed our consciousness of our inner experiences. They are more varied than just perception at the footlights of our consciousness (cf Hurlburt): Imagination, inner voice, emotion, unsymbolized conceptual thinking as well as (our reconstructed view of our) perception.

oh no, those people without an inner voice are now cowering in a corner...

Everyone has some introspection into their own thoughts, it just takes different forms.
Post reply on HN