Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

261–270 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#261

Post by crystal_revenge »

danenania wrote: post #206
crystal_revenge wrote: post #190

Earlier quoted context omitted.

The magical thinking around LLMs is getting bizarre now. LLMs are not “intelligent” in any meaningful biological sense. Watch a spider modify its web to adapt to changing conditions and you’ll realize just how far we have to go. LLMs sometimes echo our own reasoning back at us in a way that sounds intelligent and is often useful, but don’t mistake this for “intelligence”

The idea that biological intelligence is impossible to replicate by other means would seem to imply that there’s something magical about biology.

I'm nowhere implying that it's impossible to replicate, just that LLMs have almost nothing to do with replicating intelligence. They aren't doing any of the things even simple life forms are doing.

Re: AGI is an engineering problem, not a model training problem

#262

Post by _w1tm »

tomrod wrote: post #195
galangalalgol wrote: post #173

Earlier quoted context omitted.

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

I mean, I'm conscious to a degree, and can alter that state through a number of activities. I can't speak for you or Metzinger ;). But seriously, I get why free will is troubleaome, but the fact people can choose a thing, work at the thing, and effectuate the change against a set of options they had never considered before an initial moment of choice is strong and sufficient evidence against anti free will claims. It…

> But seriously, I get why free will is troubleaome, but the fact people can choose a thing, work at the thing, and effectuate the change against a set of options they had never considered before an initial moment of choice is strong and sufficient evidence against anti free will claims.

Do people choose a thing or was the thing chosen for them by some inputs they received in the past?

Re: AGI is an engineering problem, not a model training problem

#263

Post by solatic »

Ctrl-F -> emotion -> 0/0 not found in the article.

Trying to model AGI off how humans think, without including emotion as a fundamental building block, is like trying to build a computer that'll run without electricity. People are emotional beings first. So much of how we learn that something is good or bad is due to emotion.

In an AGI context that means:

  Happiness: how do I build an unguided feedback mechanism for reward?
  Fear: how do I build an unguided feedback mechanism to instruct to flee?
  Sadness: how do I build an unguided feedback mechanism to instruct to seek external support?
  Anger: how do I build an unguided feedback mechanism to push back on external entities that violate expectations?
  Disgust: how do I build an unguided feedback mechanism to instruct to avoid?
Maybe building artificial emotions is an engineering problem. Maybe not. But approaches that avoid emotion entirely seem ill-advised.

Re: AGI is an engineering problem, not a model training problem

#264

Post by emporas »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

But then it is a specialized intelligence, specialized to altering it's weights. Reinforcement Learning doesn't work as well when the goal is not easily defined. It does wonders for games, but anything else?

Someone has to specify the goals, a human operator or another A.I. The second A.I. better be an A.G.I. itself, otherwise it's goals will not be significant enough for us to care.

Re: AGI is an engineering problem, not a model training problem

#265

Post by Jensson »

galangalalgol wrote: post #173
Uehreka wrote: post #116

Earlier quoted context omitted.

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

Illusions are real things though, they aren't ghosts there is science behind them. So if they are like illusions then we can explain what it is and why we experience it that way.

Re: AGI is an engineering problem, not a model training problem

#266

Post by android521 »

ivraatiems wrote: post #231

I think I am coming to agree with the opinions of the author, at least as far as LLMs not being the key to AGI on their own. The sheer impressiveness of what they can do, and the fact they can do it with natural language, made it feel like we were incredibly close for a time. As we adjust to the technology, it starts to feel like we're further away again. But I still see all the same debates around AGI - how do we de…

because if people could do it , they would do it. And if you country decides you should not do it, you could be left behid. This possibility prevents any country from not doing it if they could unless they are willing to start wars with other countries for compliance (and they would still secretly do it). So should is an irrelevant quesiton.

Re: AGI is an engineering problem, not a model training problem

#267

Post by Jensson »

danenania wrote: post #171
fragmede wrote: post #77

Earlier quoted context omitted.

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I’m not sure that self-updating weights is really analogous to “continuous learning” as humans do it. A memory data structure that the model can search efficiently might be a lot closer. Self-updating weights could be more like epigenetics.

Human neurons are self updating though, we aren't running on our genes each cell is using our genes to determine how to connect to other cells and then the cell learns how to process some information there based on what it hears from its connected cells.

So, genes would be a meta model that then updates weights in the real model so it can learn how to process new kinds of things, and for stuff like facts you can use an external memory just like humans does.

Without updating the weights in the model you will never be able to learn to process new things like a new kind of math etc, since you learn that not by memorizing facts but by making new models for it.

Re: AGI is an engineering problem, not a model training problem

#268

Post by Jensson »

Davidzheng wrote: post #165
bsenftner wrote: post #134

Earlier quoted context omitted.

Are you sure? Do you require dozens, to hundreds, to thousands of examples before you understand a concept? I expect no. That is because you have comprehension that can generalize a situation to basic concepts which you apply to other situations without effort. You comprehend. AI cannot do that: get the idea from a few, under a half dozen examples if necessary. Often a human needs 1-3 examples before they can general…

I think they're saying people generally don't learn language or mathematics by learning the basic rules and deducing everything else

Humanity did exactly that though, so an AGI should be capable of the same feat given enough time.

Re: AGI is an engineering problem, not a model training problem

#269

Post by Jensson »

jibal wrote: post #145
bastawhiz wrote: post #85

Earlier quoted context omitted.

Well general intelligence in humans already exists, whereas general intelligence doesn't yet exist in machines. How do we know when we have it? You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be. Surely you wouldn't say that someone who can't remember names or navigate without GPS lacks general intelligence, so it's…

How do we know when a newborn has achieved general intelligence? We don't need a definition amenable to proof.

Its a near clone of a model that already has it, we don't need to prove it has general intelligence we just assume it does because most do have it.

Re: AGI is an engineering problem, not a model training problem

#270

Post by rrrrrrrrrrrryan »

MadnessASAP wrote: post #209
cakealert wrote: post #194

There is a reason why LLM's are architected the way they are and why thinking is bolted on. The architecture has to allow for gradient descent to be a viable training strategy, this means no branching (routing is bolted on). And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't…

You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output. While you say reinforcement learning isn't a good answer, I think its the only answer.

People have speculated that the main thing that sets the human mind apart from the minds of all other animals is its capacity for recursive thought. A handful of animals have been observed to use tools, but humans are the only species ever observed to use a tool to create another tool. This recursion created all of civilization.

But that recursive thought has a limit. For example: You can think about yourself thinking. With a little effort, you can probably also think about yourself thinking about yourself thinking. But you can't go much deeper.

With the advent of modern computing, we (as a species) have finally created a tool that can "think" recursively, to arbitrary levels of depth. If we ever do create a superintelligent AGI, I'd wager that its brilliance will be attributable to its ability to loop much deeper than humans can.

Post reply on HN