Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

111–120 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#111

Post by kelnos »

xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

> Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago?

That wouldn't have occurred to me, to be honest. To me, AGI is Data from Star Trek. Or at the very least, Arnold Schwarzenegger's character from The Terminator.

I'm not sure that I'd make sentience a hard requirement for AGI, but I think my general mental fantasy of AGI even includes sentience.

Claude Code is amazing, but I would never mistake it for AGI.

Re: AGI is an engineering problem, not a model training problem

#112

Post by thfuran »

throwaway31131 wrote: post #30
tshaddox wrote: post #21

Earlier quoted context omitted.

> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”

We would also need a definition of AGI that is provable or disprovable. We don’t even have a workable definition, never mind a machine.

Only if we need to classify things near the boundary. If we make something that’s better at every test that we can devise than any human we can find, I think we can say that no reasonable definition of AGI would exclude it without actually arriving at a definition.

Re: AGI is an engineering problem, not a model training problem

#113

Post by fastball »

Fricken wrote: post #95
mattnewton wrote: post #43

Earlier quoted context omitted.

Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?

We're fixated on human intelligence but a computer cannot even emulate the intelligence of a honeybee or an ant.

How do you mean? AFAICT computers can definitely do that.

Sure, it won't be the size of an ant, but we definitely have models running on computers that have much more complexity than the life of an ant.

Re: AGI is an engineering problem, not a model training problem

#114

Post by card_zero »

thfuran wrote: post #106
card_zero wrote: post #91

Earlier quoted context omitted.

There's that aphorism that goes: people who thought the epitome of technology was a steam engine pictured the brain as pipes and connecting rods, people who thought the epitome of technology was a telephone exchange pictured the brain as wires and relays... and now we have computers, and the fact that they can in principle simulate anything at all is a red herring, because we can't actually make them simulate things…

>We still need to know what the thing is that the brain does Yes, but not necessarily at the level where the interesting bits happen. It’s entirely possible to simulate poorly understood emergent behavior by simulating the underlying effects that give rise to it.

Can I paraphrase that as make an imitation and hack it around until it thinks, or did I miss the point?

Re: AGI is an engineering problem, not a model training problem

#115

Post by therobots927 »

kelnos wrote: post #105
therobots927 wrote: post #15

The problem is that if it's an engineering problem then further advancement will rely on step function discoveries like the transformer. There's no telling when that next breakthrough will come or how many will be needed to achieve AGI. In the meantime I guess all the AI companies will just keep burning compute to get marginal improvements. Sounds like a solid plan! The craziest thing about all of this is that ML res…

I think the LLM businesses as-is are potentially fine businesses. Certainly the compute cost of running and using them is very high, not yet reflected in the prices companies like OpenAI and Anthropic are charging customers. It remains to be seen if people will pay the real costs. But even if LLMs are going to tap out at some point, and are a local maximum, dead-end, when it comes to taking steps toward AGI, I would…

Anthropic isn't even breaking even, and even if they do become profitable it's a far cry from AGI

Re: AGI is an engineering problem, not a model training problem

#116

Post by Uehreka »

hnuser123456 wrote: post #47
andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

Nah.

The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain.

Instead we got a machine that can outwardly present as human, can do tasks we had thought only humans can do, but reveals little to us about the nature of consciousness. And all we can do is keep arguing about the goalposts as this thing irrevocably reshapes our society, because it seems bizarre that we could be bested by something so banal and mechanical.

Re: AGI is an engineering problem, not a model training problem

#117

Post by fuckaj »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

True. In the same way as making noises down a telephone line is the obvious way to build a million dollar business.

Re: AGI is an engineering problem, not a model training problem

#118

Post by sdenton4 »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

Too bad about all those chumps designing better, faster architectures and kernels to make models run faster...

Re: AGI is an engineering problem, not a model training problem

#119

Post by [deleted] »

kelnos wrote: post #101
tshaddox wrote: post #53

Earlier quoted context omitted.

We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.

I disagree. We claim that biological humans have general intelligence because we are biased and arrogant, and experience hubris. I'm not saying we aren't generally intelligent, but a big part of believing we are is because not believing so would be psychologically and culturally disastrous. I fully expect that, as our attempts at AGI become more and more sophisticated, there will be a long period where there are inte…

[deleted]

Re: AGI is an engineering problem, not a model training problem

#120

Post by kelnos »

Loughla wrote: post #8
sublinear wrote: post #4

Way out of touch. AGI is poorly defined and thus is a science "problem", and a very low priority one at that. No amount of engineering or model training is going to get us AGI until someone defines what properties are required and then researches what can be done to achieve them within our existing theories of computation which all computers being manufactured today are built upon.

It strikes me that until we fully understand human consciousness, we don't stand a chance of reaching AGI. Am I incorrect?

Unclear. You might be right, but I think it's possible that you're also wrong.

It's possible to stumble upon a solution to something without fully understanding the problem. I think this happens fairly often, really, in a lot of different problem domains.

I'm not sure we need to fully understand human consciousness in order to build an AGI, assuming it's possible to do so. But I do think we need to define what "general intelligence" is, and having a better understanding of what in our brains makes us generally intelligent will certainly help us move forward.

Post reply on HN