Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

11–20 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#11

Post by sixo »

glitchc wrote: post #10

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…

On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.

Re: AGI is an engineering problem, not a model training problem

#12

Post by root_axis »

Loughla wrote: post #8
sublinear wrote: post #4

Way out of touch. AGI is poorly defined and thus is a science "problem", and a very low priority one at that. No amount of engineering or model training is going to get us AGI until someone defines what properties are required and then researches what can be done to achieve them within our existing theories of computation which all computers being manufactured today are built upon.

It strikes me that until we fully understand human consciousness, we don't stand a chance of reaching AGI. Am I incorrect?

That doesn't seem like a useful assumption since consciousness doesn't have a functional definition (even though it might have a functional purpose in humans)

Re: AGI is an engineering problem, not a model training problem

#13

Post by dinobones »

Nah,this sounds like a modern remix of Japan’s Fifth Generation Computing project. They thought that by building large databases and with Prolog they would bring upon an AI renaissance.

Just hand waving some “distributed architecture” and trying to duct tape modules together won’t get us any closer to AGI.

The building blocks themselves, the foundation, has to be much better.

Arguably the only building block that LLMs have contributed is that we have better user intent understanding now; a computer can just read text and extract intent from it much better than before. But besides that, the reasoning/search/“memory” are the same building blocks of old, they look very similar to techniques of the past, and that’s because they’re limited by information theory / computer science, not by today’s hardware or systems.

Re: AGI is an engineering problem, not a model training problem

#14

Post by levitate »

"AGI needs to update beliefs when contradicted by new evidence" is a great idea, however, the article's approach of building better memory databases (basically fancier RAG) doesn't seem enable this. Beliefs and facts are built into LLMs at a very low layer during training. I wonder how they think they can force an LLM to pull from the memory bank instead of the training data.

Re: AGI is an engineering problem, not a model training problem

#15

Post by therobots927 »

The problem is that if it's an engineering problem then further advancement will rely on step function discoveries like the transformer. There's no telling when that next breakthrough will come or how many will be needed to achieve AGI.

In the meantime I guess all the AI companies will just keep burning compute to get marginal improvements. Sounds like a solid plan! The craziest thing about all of this is that ML researchers should know better!! Anyone with extensive experience training models small or large knows that additional training data offers asymptotic improvements.

Re: AGI is an engineering problem, not a model training problem

#18

Post by karmakaze »

sublinear wrote: post #4

Way out of touch. AGI is poorly defined and thus is a science "problem", and a very low priority one at that. No amount of engineering or model training is going to get us AGI until someone defines what properties are required and then researches what can be done to achieve them within our existing theories of computation which all computers being manufactured today are built upon.

This is the funniest "I'm a hammer thus AGI is a nail" post I've ever read.

Re: AGI is an engineering problem, not a model training problem

#19

Post by andy99 »

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance.

So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing science, not missing engineering.

Post reply on HN