Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

71–80 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#71

Post by gwern »

mdp2021 wrote: post #66
gwern wrote: post #52

Earlier quoted context omitted.

It is especially not obvious because this was written using ChatGPT-5. One appreciates the (deliberate?) irony, at least. (Or at least, surely if they had asymptoted, OP should've been able to write this upvoted HN article with an old GPT-4, say...)

> this was written using How do you know?

It is lacking in URLs or references. (The systematic error in the self-reference blog post URLs is also suspicious: outdated system prompt? If nothing else, shows the human involved is sloppy when every link is broken.) The assertions are broadly cliche and truisms, and the solutions are trendy buzzwords from a year ago or more (consistent with knowledge cutoffs and emphasizing mainstream sources/opinions). The tricolon and unordered bolded triplet lists are ChatGPT. The em dashes (which you should not need to be told about at this point) and it's-not-x-but-y formulation are extremely blatant, if not 4o-level, and lacking emoji or hyperbolic language; hence, it's probably GPT-5. (Sub-GPT-5 ChatGPTs would also generally balk at talking about a 'GPT-5' because they think it doesn't exist yet.) I don't know if it was 100% GPT-5-written, but I do note that when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it, and identifies several stupid assertions (eg. the claim that power law scaling has now hit 'diminishing returns', which is meaningless because all log or power laws always have diminishing returns), so probably not completely-GPT-5-written (or least, sub-Pro).

Re: AGI is an engineering problem, not a model training problem

#72

Post by mdp2021 »

levitate wrote: post #14

"AGI needs to update beliefs when contradicted by new evidence" is a great idea, however, the article's approach of building better memory databases (basically fancier RAG) doesn't seem enable this. Beliefs and facts are built into LLMs at a very low layer during training. I wonder how they think they can force an LLM to pull from the memory bank instead of the training data.

> how they think they can force an LLM to pull from the memory bank instead of the training data

You have to implement procedurality first (e.g. counting, after proper instancing of ideas).

Re: AGI is an engineering problem, not a model training problem

#74

Post by daveguy »

mdp2021 wrote: post #55
mattfrommars wrote: post #25

I don't you know about you guys but Sam Altman have said they have achieved AGI within OpenAI. That's big.

If "context is king" in the LLMs age... Well give us at least some context.

Well, they've said they're close over and over again. Maybe that final bit of tech to make AGI a reality will finally ride into existence on the sub-$30k Tesla.

Re: AGI is an engineering problem, not a model training problem

#75

Post by epiccoleman »

mmoskal wrote: post #38
epiccoleman wrote: post #23

Earlier quoted context omitted.

Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.

Consciousness (subjective experience) is possibly orthogonal to intelligence (ability to achieve complex goals). We definitely have a better handle on what intelligence is than consciousness.

That does make sense, reminds me of Blindsight, where one central idea is that conscious experience might not even be necessary for intelligence (and possibly even maladaptive).

Re: AGI is an engineering problem, not a model training problem

#76

Post by mitthrowaway2 »

bee_rider wrote: post #63
mattnewton wrote: post #43

Earlier quoted context omitted.

Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?

The post I was responding to had > On the contrary, we have one working example of general intelligence (humans) I think some animals probably have what most people would informally call general intelligence, but maybe there’s some technical definition that makes me wrong.

Their point is not in any way weakened if you read "one working example" as "at least one working example".

Re: AGI is an engineering problem, not a model training problem

#77

Post by fragmede »

hnuser123456 wrote: post #47
andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

Re: AGI is an engineering problem, not a model training problem

#78

Post by mdp2021 »

gwern wrote: post #71
mdp2021 wrote: post #66

Earlier quoted context omitted.

> this was written using How do you know?

It is lacking in URLs or references. (The systematic error in the self-reference blog post URLs is also suspicious: outdated system prompt? If nothing else, shows the human involved is sloppy when every link is broken.) The assertions are broadly cliche and truisms, and the solutions are trendy buzzwords from a year ago or more (consistent with knowledge cutoffs and emphasizing mainstream sources/opinions). The trico…

> when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it

I don't know about GPT-5-Pro, but LLMs can dislike their own output (when they work well...).

Re: AGI is an engineering problem, not a model training problem

#79

Post by soulofmischief »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

It's not always as useful as you think from the perspective of a business trying to sell an automated service to users who expect reliability. Now you have to worry about waking up in the middle of the night to rewind your model to a last known good state, leading to real data loss as far as users are concerned.

Data and functionality become entwined and basically you have to keep these systems on tight rails so that you can reason about their efficacy and performance, because any surgery on functionality might affect learned data, or worse, even damage a memory.

It's going to take a long time to solve these problems.

Re: AGI is an engineering problem, not a model training problem

#80

Post by gwern »

mdp2021 wrote: post #78
gwern wrote: post #71

Earlier quoted context omitted.

It is lacking in URLs or references. (The systematic error in the self-reference blog post URLs is also suspicious: outdated system prompt? If nothing else, shows the human involved is sloppy when every link is broken.) The assertions are broadly cliche and truisms, and the solutions are trendy buzzwords from a year ago or more (consistent with knowledge cutoffs and emphasizing mainstream sources/opinions). The trico…

> when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it I don't know about GPT-5-Pro, but LLMs can dislike their own output (when they work well...).

They can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if that was part of OP's original inputs (whatever those were) it is much less surprising (because it is a common human error and mindlessly parroted in a lot of the 'scaling has hit a wall' human journalism).
Post reply on HN