Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

101–110 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#101

Post by kelnos »

tshaddox wrote: post #53
throwaway31131 wrote: post #30

Earlier quoted context omitted.

We would also need a definition of AGI that is provable or disprovable. We don’t even have a workable definition, never mind a machine.

We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.

I disagree. We claim that biological humans have general intelligence because we are biased and arrogant, and experience hubris. I'm not saying we aren't generally intelligent, but a big part of believing we are is because not believing so would be psychologically and culturally disastrous.

I fully expect that, as our attempts at AGI become more and more sophisticated, there will be a long period where there are intensely polarizing arguments as to whether or not what we've built is AGI or not. This feels so obvious and self-evident to me that I can't imagine a world where we achieve anything approaching consensus on this quickly.

If we could come up with a widely-accepted definition of general intelligence, I think there'd be less argument, but it wouldn't preclude people from interpreting both the definition and its manifestation in different ways.

Re: AGI is an engineering problem, not a model training problem

#102

Post by kelnos »

epiccoleman wrote: post #82
tshaddox wrote: post #50

Earlier quoted context omitted.

That sounds like you’re describing AGI as being impractical to implement in an electronic computer, not impossible in principle.

Yeah, I guess I'm not taking a stance on that above, just wondering where in that chain holds the most explanatory power for intelligence and/or consciousness. I don't think there's any real reason to think intelligence depends on "meat" as its substrate, so AGI seems in principle possible to me. Not that my opinion counts for much on this topic, since I don't really have any relevant education on the topic. But my h…

> I don't think there's any real reason to think intelligence depends on "meat" as its substrate

This made me think of... ok, so let's say that we discover that intelligence does indeed depend on "meat". Could we then engineer a sort of organic computer that has general intelligence? But could we also claim that this organic computer isn't a computer at all, but is actually a new genetically engineered life form?

Re: AGI is an engineering problem, not a model training problem

#103

Post by kelnos »

DrewADesign wrote: post #59
singpolyma3 wrote: post #37

Earlier quoted context omitted.

This makes no sense. If you believe in eg a mind or soul then maybe it's possible we cannot make AGI. But if we are purely biological then obviously it's possible to replicate that in principle.

That doesn’t contradict what they said. We may one day design a biological computing system that is capable of it. We don’t entirely understand how neurons work; it’s reasonable to posit that the differences that many AGI boosters assert don’t matter do matter— just not in ways we’ve discovered yet.

I mentioned this in another thread, but I do wonder if we engineer a sort of biological computer, will it really be a computer at all, and not a new kind of life itself?

Re: AGI is an engineering problem, not a model training problem

#104

Post by furyofantares »

zdragnar wrote: post #86
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Claude code is neither sentient nor sapient. I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is. The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.

> I suspect most people envision AGI as at least having sentience

I certainly don't. It could be that's necessary but I don't know of any good arguments for (or against) it.

Re: AGI is an engineering problem, not a model training problem

#105

Post by kelnos »

therobots927 wrote: post #15

The problem is that if it's an engineering problem then further advancement will rely on step function discoveries like the transformer. There's no telling when that next breakthrough will come or how many will be needed to achieve AGI. In the meantime I guess all the AI companies will just keep burning compute to get marginal improvements. Sounds like a solid plan! The craziest thing about all of this is that ML res…

I think the LLM businesses as-is are potentially fine businesses. Certainly the compute cost of running and using them is very high, not yet reflected in the prices companies like OpenAI and Anthropic are charging customers. It remains to be seen if people will pay the real costs.

But even if LLMs are going to tap out at some point, and are a local maximum, dead-end, when it comes to taking steps toward AGI, I would still pay for Claude Code until and unless there's something better. Maybe a company like Anthropic is going to lead that research and build it, or maybe (probably) it's some group or company that doesn't exist yet.

Re: AGI is an engineering problem, not a model training problem

#106

Post by thfuran »

card_zero wrote: post #91
tshaddox wrote: post #65

Earlier quoted context omitted.

> It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Yes, that is the bluntest, lowest level version of what I mean. To discover that this wouldn’t work in principle would be to discover that quantum mechanics is false. Which, hey, quantum mechanics probably is false! But discovering the theory whic…

There's that aphorism that goes: people who thought the epitome of technology was a steam engine pictured the brain as pipes and connecting rods, people who thought the epitome of technology was a telephone exchange pictured the brain as wires and relays... and now we have computers, and the fact that they can in principle simulate anything at all is a red herring, because we can't actually make them simulate things…

>We still need to know what the thing is that the brain does

Yes, but not necessarily at the level where the interesting bits happen. It’s entirely possible to simulate poorly understood emergent behavior by simulating the underlying effects that give rise to it.

Re: AGI is an engineering problem, not a model training problem

#107

Post by solid_fuel »

ants_everywhere wrote: post #90
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

> there is missing philosophy I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.

We are trying to get there without a few hundred million years of trial and error. To do that we need to lower the search space, and to do that we do actually need more guiding philosophy and a better understanding of intelligence.

Re: AGI is an engineering problem, not a model training problem

#108

Post by handfuloflight »

zdragnar wrote: post #86
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Claude code is neither sentient nor sapient. I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is. The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.

Why should it have discrete identity and notion of self?

Re: AGI is an engineering problem, not a model training problem

#109

Post by jakeydus »

kelnos wrote: post #103
DrewADesign wrote: post #59

Earlier quoted context omitted.

That doesn’t contradict what they said. We may one day design a biological computing system that is capable of it. We don’t entirely understand how neurons work; it’s reasonable to posit that the differences that many AGI boosters assert don’t matter do matter— just not in ways we’ve discovered yet.

I mentioned this in another thread, but I do wonder if we engineer a sort of biological computer, will it really be a computer at all, and not a new kind of life itself?

> not a new kind of life itself?

In my opinion, this is more a philosophical question than an engineering one. Is something alive because it’s conscious? Is it alive because it’s intelligent? Is a virus alive, or a bacteria, or an LLM?

Beats me.

Re: AGI is an engineering problem, not a model training problem

#110

Post by fuckaj »

ants_everywhere wrote: post #90
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

> there is missing philosophy I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.

The physical universe has much higher throughput and lower latency than our computer emulating a digital world.
Post reply on HN