Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

91–100 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#91

Post by card_zero »

tshaddox wrote: post #65
AIPedant wrote: post #56

Earlier quoted context omitted.

It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Obviously this is cost-prohibitive and we don’t have even 0.1% of the data required to make the simulation. Maybe we could simulate every single neuron instead, but again it’ll take many decades to gather the data in living human brains, and it woul…

> It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Yes, that is the bluntest, lowest level version of what I mean. To discover that this wouldn’t work in principle would be to discover that quantum mechanics is false. Which, hey, quantum mechanics probably is false! But discovering the theory whic…

There's that aphorism that goes: people who thought the epitome of technology was a steam engine pictured the brain as pipes and connecting rods, people who thought the epitome of technology was a telephone exchange pictured the brain as wires and relays... and now we have computers, and the fact that they can in principle simulate anything at all is a red herring, because we can't actually make them simulate things we don't understand, and we can't always make them simulate things we do understand, either, when it comes down to it. We still need to know what the thing is that the brain does, it's still a hard question, and maybe it would even be a kind of revolution in physics, just not in fundamental physics.

Re: AGI is an engineering problem, not a model training problem

#92

Post by nativeit »

Say we get there--all the way, full AGI, indistinguishable from conscious intelligence. Congratulations, every thing you do from that point on (very likely everything from well before that point) that is not specifically granting it free will and self-determination is slavery. Doesn't really feel like a real "win" for any plausible end goal. I'm really not clear on why anyone thinks this is a good idea, or desirable, let alone possible?

Re: AGI is an engineering problem, not a model training problem

#93

Post by tshaddox »

bastawhiz wrote: post #85
tshaddox wrote: post #53

Earlier quoted context omitted.

We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.

Well general intelligence in humans already exists, whereas general intelligence doesn't yet exist in machines. How do we know when we have it? You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be. Surely you wouldn't say that someone who can't remember names or navigate without GPS lacks general intelligence, so it's…

> You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be.

Right, but you can’t compare two different humans either. You don’t test each new human to see if they have it. Somehow we conclude that humans have it without doing either of those things.

Re: AGI is an engineering problem, not a model training problem

#94

Post by ramoz »

exabrial wrote: post #87

https://mashable.com/article/apple-research-ai-reasoning-mod... All of our current approaches "emulate" but do not "execute" general intelligence. The damning paper above basically concludes they're incredible pattern matching machines, but thats about it.

We’ve not determined whether or not that isn’t a useful mechanism for capable intelligence.

For instance it is becoming clearer that you can build harnesses for a well-trained model and teach it how to use that harness in conjunction with powerful in-context learning. I’m explicitly speaking of the Claude models and the power of whatever it is they started doing in RL. Truly excited to see where they take things and the continued momentum with tools like Claude Code (a production harness).

Re: AGI is an engineering problem, not a model training problem

#95

Post by Fricken »

mattnewton wrote: post #43
bee_rider wrote: post #27

Earlier quoted context omitted.

Do we have a specific enough definition of general intelligence that we can exclude all non-human animals?

Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?

We're fixated on human intelligence but a computer cannot even emulate the intelligence of a honeybee or an ant.

Re: AGI is an engineering problem, not a model training problem

#96

Post by kelseyfrog »

zdragnar wrote: post #86
xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Claude code is neither sentient nor sapient. I suspect most people envision AGI as at least having sentience. To borrow from Star Trek, the Enterprise's main computer is not at the level of AGI, but Data is. The biggest thing that is missing (IMHO) is a discrete identity and notion of self. It'll readily assume a role given in a prompt, but lacks any permanence.

Mine is. What evidence would you accept to change your mind?

Re: AGI is an engineering problem, not a model training problem

#97

Post by arcanemachiner »

nativeit wrote: post #92

Say we get there--all the way, full AGI, indistinguishable from conscious intelligence. Congratulations, every thing you do from that point on (very likely everything from well before that point) that is not specifically granting it free will and self-determination is slavery. Doesn't really feel like a real "win" for any plausible end goal. I'm really not clear on why anyone thinks this is a good idea, or desirable,…

You can draw whatever arbitrary line you want, and say "anything on the other side of this line is slavery", but that doesn't mean that your supposition is true.

Re: AGI is an engineering problem, not a model training problem

#98

Post by Foreignborn »

gwern wrote: post #80
mdp2021 wrote: post #78

Earlier quoted context omitted.

> when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it I don't know about GPT-5-Pro, but LLMs can dislike their own output (when they work well...).

They can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if that was part of OP's original inputs (whatever those were) it is much less surprising (because it is a common human error and mindlessly parroted in a lot of the 'scaling has hit a w…

do you have a source?

when i’ve done toy demos where GPT5, sonnet 4 and gemini 2.5 pro critique/vote on various docs (eg PRDs) they did not choose their own material more often than not.

my setup wasn’t intended to benchmark though so could be wrong over enough iterations.

Re: AGI is an engineering problem, not a model training problem

#99

Post by tshaddox »

bilsbie wrote: post #89
tshaddox wrote: post #21

Earlier quoted context omitted.

> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”

Not necessarily. It could simply be a question of scale. Being analog and molecular means that brain could be doing enormously more than any foreseeable computer. For a simple example what if every neuron is doing trillions of calculations. (I’m not saying it is, just that it’s possible)

I think you’re merely referring to what is feasible in practice to computer with our current or near-future computers. I was referring to what is computable in principle.

Re: AGI is an engineering problem, not a model training problem

#100

Post by [deleted] »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

[deleted]
Post reply on HN