Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

361–370 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#361

Post by amanaplanacanal »

ACCount37 wrote: post #348
amanaplanacanal wrote: post #346

Earlier quoted context omitted.

"Consciousness" seems pretty obvious. The ability to experience qualia. I do it, you do it, my dog does it. I suspect all mammals do it, and I suspect birds do too. There is no evidence any computer program does anything like it. It's "intelligence" I can't define.

Oh, so simple. Go measure it then. The definition of "featherless biped" might have more practical merit, because you can at least check for feathers and count limbs touching the ground in a mostly reliable fashion. We have no way to "check for qualia" at all. For all we know, an ECU in a year 2002 Toyota Hilux has it, but 10% of all humans don't.

Plenty of things are real that can't be measured, including many physical sensations and emotions.

I won't say they are impossible to ever be measured, but we currently have no idea how.

Re: AGI is an engineering problem, not a model training problem

#362

Post by whatisthiseven »

Uehreka wrote: post #116
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

> The real philosophical headache

Isn't the real actual headache whether to produce another thinking intelligent being at all, and what the ramifications of that decision are? Not whether it would destroy humanity, but what it would mean for a mega corporation whose goal is to extract profit to own the rights of creating a thinking machine that identifies itself as thinking and a "self"?

Really out here missing the forest for the mushrooms growing on the trees. Or maybe this is debated to death and no one cares for the answer: its just not interesting to think about because its going to happen anyway. Might as well join the bandwagon and be along the front-lines of the bikini atoll to witness death itself be born, digitally.

Re: AGI is an engineering problem, not a model training problem

#363

Post by JRR_214 »

nikkwong wrote: post #164
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

I found it strange that John Carmack and Ilya Sutskever both left prestigious positions within their companies to pursue AGI as if they had some proprietary insight that the rest of industry hadn't caught on to. To make as bold of a career move that publicly would mean you'd have to have some ultra serious conviction that everyone else was wrong or naive and you were right. That move seemed pompous to me at the time;…

[dead]

Re: AGI is an engineering problem, not a model training problem

#364

Post by bitwize »

zdragnar wrote: post #86
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Claude code is neither sentient nor sapient. I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is. The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.

The analogy I like to use is from the fictional universe of Mass Effect, which distinguished between VI (Virtual Intelligence), which is a conversational interface over some database or information service (often with a holographic avatar of a human, asari, or other sentient being); and AI, which is sentient and smart enough to be considered a person in its own right. We've just barely begun to construct VIs, and they're not particularly good or reliable ones.

One thing I like about the Mass Effect universe is the depiction of the geth, which qualify as AI. Each geth unit is not run by a singular intelligent program, but rather a collection of thousands of daemons, each of which makes some small component of the robot's decisions on its own, but together they add up to a collective consciousness. When you look at how actual modern robotics platforms (such as ROS) are designed, with many processes responsible for sensors and actuators communicating across a common bus, you can see the geth as sort of an extrapolation of that idea.

Re: AGI is an engineering problem, not a model training problem

#365

Post by dragonwriter »

Jensson wrote: post #268
Davidzheng wrote: post #165

Earlier quoted context omitted.

I think they're saying people generally don't learn language or mathematics by learning the basic rules and deducing everything else

Humanity did exactly that though, so an AGI should be capable of the same feat given enough time.

> Humanity did exactly that though

No, it mostly didn't, it continued (continues, as every human is continuously interlacing “training” and “inferencing”) training on large volumes of ground truth for a very long time, including both natural and synthetic data; it didn't reason everything beyond some basic training on first principles.

At a minimum, something that looks broadly like one of today's AI models would need either a method of continuously finetuning its own weights with a suitable evaluation function or,if it was going to rely on in-context learning, would need many orders of magnitude larger context, than any model today.

And that's not a “this is enough to likely work” thing, but “this is the minimum for the their to even be a plausible mechnanism to incorporate the information necessary for it to work” one.

Re: AGI is an engineering problem, not a model training problem

#366

Post by CuriouslyC »

Uehreka wrote: post #116
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

We have a pretty obvious solution to the hard problem. Panpsychism. People are just afraid of the idea.

Re: AGI is an engineering problem, not a model training problem

#367

Post by CuriouslyC »

galangalalgol wrote: post #173
Uehreka wrote: post #116

Earlier quoted context omitted.

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

Pretty sure the truth is exactly the opposite. Conscious is real, and this reality you're playing in is the virtual construct.

Re: AGI is an engineering problem, not a model training problem

#368

Post by dragonwriter »

dns_snek wrote: post #278
idiotsecant wrote: post #243

Earlier quoted context omitted.

You spend every waking minute for 20 years or so accumulating training data. You don't learn addition and then independently discover vector calculus.

Individual people don't but we did it as a species. Any purported AGI should be capable of doing the same.

So you are now claiming that individual humans are not general intelligences and the only natural general intelligence is humanity as a unit?

Re: AGI is an engineering problem, not a model training problem

#369

Post by marcosdumay »

tshaddox wrote: post #61
marcosdumay wrote: post #42

Earlier quoted context omitted.

> Would that really be a physics discovery? No, it could be something that proves all of our fundamental mathematics wrong. The GP just gave the more conservative option.

I’m not sure what you mean. This new discovery in mathematics would also necessarily tell us something new about what is computable, which is physics.

It would impact physics, yes. And literally every other natural science.

Re: AGI is an engineering problem, not a model training problem

#370

Post by taormina »

danenania wrote: post #193
taormina wrote: post #136

Earlier quoted context omitted.

Just ancedata, but they keep releasing new versions and it keeps not being better. What would you describe this as if not plateauing? Worsening?

I see a lot of people saying things like this, and I’m not really sure which planet you all are living on. I use LLMs nearly every day, and they clearly keep getting better.

Grok hasn't gotten better. OpenAI hasn't gotten better. Claude Code with Opus and Sonnet I swear are getting actively worse. Maybe you only use them for toy projects, but attempting to get them to do real work in my real codebase is an exercise in frustration. Yes, I've done meaningful prompting work, and I've set up all the CLAUDE.md files, and then it proceeds to completely ignores everything I said, all of the context I gave, and just craps out something completely useless. It has accomplished a small amount of meaningful work, exactly enough that I think I'm neutral instead of in the negative in terms of work:time if I have just done it all myself.

I get to tell myself that it's worth it because at least I'm "keeping up with the industry" but I honestly just don't get the hype train one bit. Maybe I'm too senior? Maybe the frameworks I use, despite being completely open source and available as training data for every model on the planet are too esoteric?

And then the top post today on the front page is telling me that my problem is that I'm bothering to supervise and that I should be writing an agent framework so that it can spew out the crap in record time..... But I need to know what is absolute garbage and what needs to be reverted. I will admit that my usual pattern has been to try and prompt it into better test coverage/specific feature additions/etc on the nights and weekends, and then I focus my daytime working hours on reviewing what was produced. About half the time I review it and have to heavily clean it up to make it usable, but more often than not, I revert the whole thing and just start on it myself from scratch. I don't see how this counts as "better".

Post reply on HN