Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

311–320 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#311

Post by chongli »

galangalalgol wrote: post #173
Uehreka wrote: post #116

Earlier quoted context omitted.

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

What does the “illusion of consciousness” mean? Sounds like question-begging to me. The word illusion presupposes a conscious being to experience it.

Machines do not experience illusions. They may have sensory errors that cause them to misbehave but they lack the subjective experience of illusion.

Re: AGI is an engineering problem, not a model training problem

#312

Post by imtringued »

fragmede wrote: post #77
hnuser123456 wrote: post #47

Earlier quoted context omitted.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with,…

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I wonder when there will be proofs in theoretical computer science that an algorithm is AGI-complete, the same way there are proofs of NP-completeness.

Conjecture: A system that self updates its weights according to a series of objective functions, but does not suffer from catastrophic forgetting (performance only degrades due to capacity limits, rather than from switching tasks) is AGI-complete.

Why? Because it could learn literally anything!

Re: AGI is an engineering problem, not a model training problem

#313

Post by tim333 »

andy99 wrote: post #19

If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…

The bitter lesson was "general methods that leverage computation" win rather than more data. Like rather than just LLMs you could maybe try applying something like AlphaEvolve to finding better algorithms/systems (https://news.ycombinator.com/item?id=43985489).

Re: AGI is an engineering problem, not a model training problem

#314

Post by imtringued »

danenania wrote: post #171
fragmede wrote: post #77

Earlier quoted context omitted.

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I’m not sure that self-updating weights is really analogous to “continuous learning” as humans do it. A memory data structure that the model can search efficiently might be a lot closer. Self-updating weights could be more like epigenetics.

In spiking neural networks, the model weights are equivalent to dendrites/synapses, which can form anew and decay during your lifetime.

Re: AGI is an engineering problem, not a model training problem

#315

Post by tim333 »

solid_fuel wrote: post #107
ants_everywhere wrote: post #90

Earlier quoted context omitted.

> there is missing philosophy I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.

We are trying to get there without a few hundred million years of trial and error. To do that we need to lower the search space, and to do that we do actually need more guiding philosophy and a better understanding of intelligence.

If you look at AI systems that have worked like chess and go programs and LLMs, they came from understanding the problems and engineering approaches but not really philosophy.

Re: AGI is an engineering problem, not a model training problem

#316

Post by tim333 »

crystal_revenge wrote: post #190
ants_everywhere wrote: post #90

Earlier quoted context omitted.

> there is missing philosophy I doubt it. Human intelligence evolved from organisms much less intelligent than LLMs and no philosophy was needed. Just trial and error and competition.

The magical thinking around LLMs is getting bizarre now. LLMs are not “intelligent” in any meaningful biological sense. Watch a spider modify its web to adapt to changing conditions and you’ll realize just how far we have to go. LLMs sometimes echo our own reasoning back at us in a way that sounds intelligent and is often useful, but don’t mistake this for “intelligence”

They pass human intelligence tests like exams and IQ tests.

If I ask chatgpt how to get rid of spiders I'm probably going to get further than the spiders would scheming to get rid of chatgpt.

Re: AGI is an engineering problem, not a model training problem

#317

Post by jsnell »

> GPT-5, Claude, and Gemini represent remarkable achievements, but they’re hitting asymptotes

This part could do with sourcing. I think it seems clearly untrue. We only have three types of benchmark: a) ones that have been saturated, b) ones where AI performance is progressing rapidly, c) really newly introduced ones that were specifically designed for the then-current frontier models to fail on. Look at for example the METR long time horizon task benchmark, which is one that's particularly resistant to saturation.

The entire article is claimed on this unsupported but probably untrue claim, but it's a bit hard to talk about when we don't have any clue about why the author thinks this is true.

> The path to artificial general intelligence isn’t through training ever-larger language models

Then it's a good thing that it's not the path most of the frontier labs are taking. It appears to be what xAI is doing for everything, and it was probably what GPT-4.5 was. Neither is a particularly compelling success story. But all the other progress over the last 12-18 months has come from models the same size or smaller advancing the frontier. And it has come from exactly the kind of engineering improvements that the author claims need to happen, both of the models and the scaffolding around the models. (RL on chain of thought, synthetic data, distillation, model-routing, tool use, subagents).

Sorry, no, they're not exactly the same kind of engineering improvements. They're the kind of engineering improvements that the people actually creating these systems though would be useful and actually worked. We don't see the failed experiments, and we don't see the ideas that weren't well-baked enough to even experiment on.

Post reply on HN