Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

281–290 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#281

Post by Mikhail_Edoshin »

danenania wrote: post #230
charcircuit wrote: post #217

Earlier quoted context omitted.

There very well could be something magical about it.

It’s fine to think that—many clearly do. But it would be more honest and productive imo if people would just say outright when they don’t think AGI is possible (or that AI can never be “real intelligence”) for religious reasons, rather than pretending there’s a rational basis.

When you try to solve a problem the goal or the reason to reject the current solution are often vague and hard to put in words. Irrational. For example, for many years the fifth postulate of Euclid was a source of mathematical discontent because of a vague feeling that it was way too complex compared to the other four. Such irrationality is a necessary step in human thought.

Re: AGI is an engineering problem, not a model training problem

#282

Post by mofeien »

Jensson wrote: post #268
Davidzheng wrote: post #165

Earlier quoted context omitted.

I think they're saying people generally don't learn language or mathematics by learning the basic rules and deducing everything else

Humanity did exactly that though, so an AGI should be capable of the same feat given enough time.

There is the concept of n-t-AGI, which is capable of performing tasks that would take n humans t time. So a single AI system that is capable of rediscovering much of science from basic principles could be classified as something like 10'000'000humans-2500years-AGI, which could already be reasonably considered artificial superintelligence.

Re: AGI is an engineering problem, not a model training problem

#284

Post by lelanthran »

rrrrrrrrrrrryan wrote: post #270
MadnessASAP wrote: post #209

Earlier quoted context omitted.

You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output. While you say reinforcement learning isn't a good answer, I think its the only answer.

People have speculated that the main thing that sets the human mind apart from the minds of all other animals is its capacity for recursive thought. A handful of animals have been observed to use tools, but humans are the only species ever observed to use a tool to create another tool. This recursion created all of civilization. But that recursive thought has a limit. For example: You can think about yourself thinkin…

> With the advent of modern computing, we (as a species) have finally created a tool that can "think" recursively, to arbitrary levels of depth.

I don't know what this means; when a computer "thinks" recursively, does it actually?

The recursion is specified by the operator (i.e. programmer), so the program that is "thinking" recursively is not, because the both the "thinking" and the recursion is provided by the tool user (the programmer), not by the tool.

> If we ever do create a superintelligent AGI, I'd wager that its brilliance will be attributable to its ability to loop much deeper than humans can.

Agreed.

Re: AGI is an engineering problem, not a model training problem

#285

Post by aubanel »

The premise "LLMs have reached a plateau" is false IMO.

Here are the metrics by which the author defines this plateau: "limited by their inability to maintain coherent context across sessions, their lack of persistent memory, and their stochastic nature that makes them unreliable for complex multi-step reasoning."

If you try to benchmark any proxy of the points above, for instance "can models solve problems that require multi steps in agentic mode" (PlanBench, BrowseComp, I've even built custom benchmarks), the progress between models is very clear, and shows no sign of slowing down.

And this does convert to real-world tasks : yesterday, I had GPT-5 build me complex react charts in one-shot, whereas previous models needed more constant supervision.

I think we're moving goalposts too fast for LLMs, that's what can lead us to believe they've plateaued : but just try using past models for your current tasks (you can use use open models to be sure they were not updated) and see them struggle.

Re: AGI is an engineering problem, not a model training problem

#286

Post by black_knight »

rrrrrrrrrrrryan wrote: post #270
MadnessASAP wrote: post #209

Earlier quoted context omitted.

You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output. While you say reinforcement learning isn't a good answer, I think its the only answer.

People have speculated that the main thing that sets the human mind apart from the minds of all other animals is its capacity for recursive thought. A handful of animals have been observed to use tools, but humans are the only species ever observed to use a tool to create another tool. This recursion created all of civilization. But that recursive thought has a limit. For example: You can think about yourself thinkin…

Off topic, but I remember as a child I would play around with that kind of recursive thinking. I would think about something, then think about that I thought about it, then think about that I though about thinking about it. Then, after a few such repetitions I would recognise that this could go on forever. Then I would think about the fact that I recognise that this could go on forever, then think about that… then realise that this meta pattern could go on forever. Etc…

Later I connected this game with the ordinals. 0,1,2… ω, ω+1, ω+2,…,2ω,2ω+1,2ω+2,…,3ω,…,4ω,…,4ω,…, ω*ω,…

Re: AGI is an engineering problem, not a model training problem

#287

Post by adastra22 »

MadnessASAP wrote: post #209
cakealert wrote: post #194

There is a reason why LLM's are architected the way they are and why thinking is bolted on. The architecture has to allow for gradient descent to be a viable training strategy, this means no branching (routing is bolted on). And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't…

You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output. While you say reinforcement learning isn't a good answer, I think its the only answer.

LLMs have loops. The output is fed back in for the next prediction cycle. How is that not the same thing?

Re: AGI is an engineering problem, not a model training problem

#288

Post by adastra22 »

Jensson wrote: post #268
Davidzheng wrote: post #165

Earlier quoted context omitted.

I think they're saying people generally don't learn language or mathematics by learning the basic rules and deducing everything else

Humanity did exactly that though, so an AGI should be capable of the same feat given enough time.

Humanity did it through A LOT of collective trial and error. Evolution is a powerful algorithm, but not a very smart one.

Re: AGI is an engineering problem, not a model training problem

#289

Post by samrus »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

> If you believe the bitter lesson, all the handwavy "engineering" is better done with more data

Id say better model architechture than more data. A human can learn to do things more complex than an LLM with less data. I think modelling the world as a static system to be representation learned in an unsupervised fashion is blocked on the static assumption. The world is dynamical, that should be reflected in the base model

But yeah, definitely not an engineering problem. Thats like saying the reason a crow isnt as smart as a person is becauss they dont have the hands to type of keyboards. But its also not because they havent seen enough of the world like your saying. Its be ause their brain isnt complex enough

Re: AGI is an engineering problem, not a model training problem

#290

Post by adastra22 »

sixo wrote: post #11
glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.

There are many working quantum computers…
Post reply on HN