Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

121–130 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#121

Post by gwern »

Foreignborn wrote: post #98
gwern wrote: post #80

Earlier quoted context omitted.

They can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if that was part of OP's original inputs (whatever those were) it is much less surprising (because it is a common human error and mindlessly parroted in a lot of the 'scaling has hit a w…

do you have a source? when i’ve done toy demos where GPT5, sonnet 4 and gemini 2.5 pro critique/vote on various docs (eg PRDs) they did not choose their own material more often than not. my setup wasn’t intended to benchmark though so could be wrong over enough iterations.

I don't have any particularly canonical reference I'd cite here, but self-preference bias in LLMs is well-established. (Just search Arxiv.)

Re: AGI is an engineering problem, not a model training problem

#122

Post by VerminOctopus1 »

The forgone conclusion that LLMs are the key or even a major step towards AGI is frustrating. They are not, and we are fooling ourselves. They are incredible knowledge stores and statistical machines, but general intelligence is far more than these attributes.

Re: AGI is an engineering problem, not a model training problem

#123

Post by kelnos »

Loughla wrote: post #81
mdp2021 wrote: post #68

Earlier quoted context omitted.

Intelligence (solving problems) does not require consciousness.

Please elaborate.

I'm not sure I'd give such an absolute statement of certainty as the GP, but there is little reason to believe that consciousness and intelligence need to go hand-in-hand.

On top of that, we don't really have good, strong definitions of "consciousness" or "general intelligence". We don't know what causes either to emerge from a complex system. We don't know if one is required to have the other (and in which direction), or if you can have an unintelligent consciousness or an unconscious intelligence.

Re: AGI is an engineering problem, not a model training problem

#125

Post by bsenftner »

AGI, by definition, in its name Artificial General Intelligence implies / directly states that this type of AI is not some dumb AI that requires training for all its knowledge, a general intelligence merely needs to be taught how to count, the basic rules of logic, and the basic rules of a single human language. From those basics all derivable logical human sciences will be rediscovered by that AGI and our next job is synchronizing with it our names for all the phenomenon that the AGI had to name on its own when that AGI self developed all the logical ramifications of our basics.

What is that? What could merely require light elementary education and then it takes off and self improves to match and surpass us? That would be artificial comprehension, something we've not even scratched. AI and trained algorithms are "universal solvers" given enough data, This AGI would be something different, this is understanding, comprehending. Instantaneous decomposition of observations for assessment of plausibility, and then recombination for assessment of combination plausibility - all continual and instant for assessment of personal safety: all that happens in people continually while awake. Be that monitoring of personal safety be for physical or loss of client during sales negotiation. Our comprehending skills are both physical and abstract. This requires a dynamic assessment, an ongoing comprehension that is validating observations as a foundation floor, so a more forward train of thought, a "conscious mind" can make decisions without conscious thought about lower level issues like situational safety. AGI needs all that dynamic comprehending capability, to satisfy its name of being general.

Re: AGI is an engineering problem, not a model training problem

#126

Post by energy123 »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

What will it scale up to if not AGI? OpenAI has a synthetic data flywheel. What are the asymptotics of this flywheel assuming no qualitative additional breakthrough?

Re: AGI is an engineering problem, not a model training problem

#128

Post by dragonwriter »

bsenftner wrote: post #125

AGI, by definition, in its name Artificial General Intelligence implies / directly states that this type of AI is not some dumb AI that requires training for all its knowledge, a general intelligence merely needs to be taught how to count, the basic rules of logic, and the basic rules of a single human language. From those basics all derivable logical human sciences will be rediscovered by that AGI and our next job i…

> AGI, by definition, in its name Artificial General Intelligence implies / directly states that this type of AI is not some dumb AI that requires training for all its knowledge, a general intelligence merely needs to be taught how to count, the basic rules of logic, and the basic rules of a single human language. From those basics all derivable logical human sciences will be rediscovered by that AGI

That's not how natural general intelligences work, though.

Re: AGI is an engineering problem, not a model training problem

#130

Post by martinald »

xyzzy123 wrote: post #31

Am I the only one who feels that Claude Code is what they would have imagined basic AGI to be like 10 years ago? It can plan and take actions towards arbitrary goals in a wide variety of mostly text-based domains. It can maintain basic "memory" in text files. It's not smart enough to work on a long time horizon yet, it's not embodied, and it has big gaps in understanding. But this is basically what I would have expec…

Totally agree. It even (usually) gets subtle meanings from my often hastily written prompts to fix something.

What really occurs to me is that there is still so much can be done to leverage LLMs with tooling. Just small things in Claude Code (plan mode for example) make the system work so much better than (eg) the update from Sonnet 3.5 to 4.0 in my eyes.

Post reply on HN