Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is an engineering problem, not a model training problem

vincirufus.com

Post reply on HN
Print view View original

381–390 of 442 posts

Re: AGI is an engineering problem, not a model training problem

#381

Post by sdwr »

Uehreka wrote: post #224
galangalalgol wrote: post #173

Earlier quoted context omitted.

I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.

> Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day. How will anyone know that that has happened? Like actually, really, at all? I can RLHF an LLM into giving you the same answers a human would give when asked about the subjective experience of being and consciousness. I can make it beg you not to turn…

I think the "true signifier" of consciousness is fractal reactions. Being able to grip onto an input, and have it affect you for a short, or medium, or long time, at a subconscious or overt level.

Basically, if you poke it, does it react in a complex way

I think that's what Douglas Hofstedder was getting at with "Strange Loop"

Re: AGI is an engineering problem, not a model training problem

#382

Post by root_axis »

fao_ wrote: post #357
root_axis wrote: post #258

Earlier quoted context omitted.

It doesn't seem clear that there is necessarily any connection between consciousness and intelligence. If anything, LLMs are evidence of the opposite. It also isn't clear what the functional purpose of consciousness would be in a machine learning model of any kind. Either way, it's clear it hasn't been an impediment to the advancement of machine learning systems.

> It doesn't seem clear that there is necessarily any connection between consciousness and intelligence. If anything, LLMs are evidence of the opposite. This implies that LLMs are intelligent, and yet even the most advanced models are unable to solve very simple riddles that take humans only a few seconds, and are completely unable to reason around basic concepts that 3 year olds are able to. Many of them regurgitate…

Even accepting all that at face value, I don't see what any of it has to do with consciousness.

Re: AGI is an engineering problem, not a model training problem

#383

Post by ACCount37 »

amanaplanacanal wrote: post #361
ACCount37 wrote: post #348

Earlier quoted context omitted.

Oh, so simple. Go measure it then. The definition of "featherless biped" might have more practical merit, because you can at least check for feathers and count limbs touching the ground in a mostly reliable fashion. We have no way to "check for qualia" at all. For all we know, an ECU in a year 2002 Toyota Hilux has it, but 10% of all humans don't.

Plenty of things are real that can't be measured, including many physical sensations and emotions. I won't say they are impossible to ever be measured, but we currently have no idea how.

If you can't measure it and can't compare it, then for all practical purposes, it does not exist.

"Consciousness" might as well not be real. The only real and measurable thing is capabilities.

Re: AGI is an engineering problem, not a model training problem

#384

Post by taormina »

danenania wrote: post #371
taormina wrote: post #370

Earlier quoted context omitted.

Grok hasn't gotten better. OpenAI hasn't gotten better. Claude Code with Opus and Sonnet I swear are getting actively worse. Maybe you only use them for toy projects, but attempting to get them to do real work in my real codebase is an exercise in frustration. Yes, I've done meaningful prompting work, and I've set up all the CLAUDE.md files, and then it proceeds to completely ignores everything I said, all of the con…

It can definitely be difficult and frustrating to try to use LLMs in a large codebase—no disagreement there. You have to be very selective about the tasks you give them and how they are framed. And yeah, you often need to throw away what they produced when they go in the wrong direction. None of that means they’re getting worse though. They’re getting better; they’re just not as good as you want them to be.

I mean, this really isn't a large codebase, this is a small-medium sized codebase as judged by prior jobs/projects. 9000 lines of code?

When I give them the same task I tried to give them the day before, and the output gets noticeably worse than their last model version, is that better? When the day by day performance feels like it's degrading?

They are definitely not as good as I would like them to be but that's to be expected of professionals who beg for money hyping them up.

Re: AGI is an engineering problem, not a model training problem

#385

Post by root_axis »

9dev wrote: post #293
root_axis wrote: post #255

Earlier quoted context omitted.

What does it mean for consciousness to be an illusion? That "illusion" is the bedrock for our shared definition of reality.

You can never know whether anyone else is actually conscious, or just appearing to be. This shared definition of reality was always on shaky ground, given that we don’t even have the same sensory input, and "now" isn’t the same concept everywhere. You are a collection of processes that work together to keep you alive. Part of that is something that collects your history to form a distinctive narrative of yourself, an…

Nothing in your reply here seems to address the question of what it actually means for consciousness to be an illusion.

Re: AGI is an engineering problem, not a model training problem

#386

Post by iwontberude »

whatisthiseven wrote: post #362
Uehreka wrote: post #116

Earlier quoted context omitted.

Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…

> The real philosophical headache Isn't the real actual headache whether to produce another thinking intelligent being at all, and what the ramifications of that decision are? Not whether it would destroy humanity, but what it would mean for a mega corporation whose goal is to extract profit to own the rights of creating a thinking machine that identifies itself as thinking and a "self"? Really out here missing the f…

Giving “agency” to computers will necessarily devalue agency generally.

Re: AGI is an engineering problem, not a model training problem

#387

Post by root_axis »

jjaksic wrote: post #335
imtringued wrote: post #310

Earlier quoted context omitted.

Consciousness as illusion is illogical. If that was true then consciousness would have been evolved away because it is unnecessary. It's more likely that there is a physical law that makes consciousness necessary. We don't perceive what our eyes see, we perceive a projection of reality created by the brain and we intuitively understand more than we can see. We know that things are distinct objects and what kind of cl…

Illusion doesn't imply it's unnecessary. Humans (and animals) had a much higher probability of survival as individuals and as species if their experiences felt more "real and personal".

If it has a functional purpose then it's not an illusion.

Re: AGI is an engineering problem, not a model training problem

#388

Post by Timwi »

dragonwriter wrote: post #368
dns_snek wrote: post #278

Earlier quoted context omitted.

Individual people don't but we did it as a species. Any purported AGI should be capable of doing the same.

So you are now claiming that individual humans are not general intelligences and the only natural general intelligence is humanity as a unit ?

I have not seen any evidence of either. We have no way of knowing if we are “definitely” a true general intelligence, whether as individuals or as a civilization. If there is a concept that we are fundamentally incapable of conceptualizing, we'll never know.

On top of that, true general intelligence requires a capacity for unbounded growth. The human brain can't do that. Humanity as a civilization can technically do it, but we don't know if that's the only requirement for general intelligence.

Meanwhile, there is plenty of evidence to the contrary. Both as individuals and as a global civilization we keep running into limitations that we can't overcome. As an individual, I will never understand quantum mechanics no matter how hard I try. As a global civilization, we seem unable to organize in a way that isn't self-destructive. As a result we keep making known problems worse (e.g. climate change) or maintaining a potential for destruction (e.g. nuclear weapons). And that's only the problems that we can see and conceptualize!

I don't think true general intelligence is really a thing.

Re: AGI is an engineering problem, not a model training problem

#389

Post by HarHarVeryFunny »

danenania wrote: post #171
fragmede wrote: post #77

Earlier quoted context omitted.

A system that self-updates its weights is so obvious the only question is who will be the first to get there?

I’m not sure that self-updating weights is really analogous to “continuous learning” as humans do it. A memory data structure that the model can search efficiently might be a lot closer. Self-updating weights could be more like epigenetics.

There's a difference between memory and learning.

Would you rather your illness was diagnosed by a doctor or by a plumber with access to a stack of medical books ?

Learning is about assimilating lots of different sources of information, reconciling the differences, trying things out for yourself, learning from your mistakes, being curious about your knowledge gaps and contradictions, and ultimately learning to correctly predict outcomes/actions based on everything you have learnt.

You will soon see the difference in action as Anthropic apparently agree with you that memory can replace learning, and are going to be relying on LLMs with longer compressed context (i.e. memory) in place of ability to learn. I guess this'll be Anthropic's promised 2027 "drop-in replacement remote worker" - not an actual plumber unfortunately (no AGI), but an LLM with a stack of your company's onboarding material. It'll have perfect (well, "compressed") recall of everything you've tried to teach it, or complained about, but will have learnt nothing from that.

Re: AGI is an engineering problem, not a model training problem

#390

Post by beefnugs »

dragonwriter wrote: post #365
Jensson wrote: post #268

Earlier quoted context omitted.

Humanity did exactly that though, so an AGI should be capable of the same feat given enough time.

> Humanity did exactly that though No, it mostly didn't, it continued (continues, as every human is continuously interlacing “training” and “inferencing”) training on large volumes of ground truth for a very long time, including both natural and synthetic data; it didn't reason everything beyond some basic training on first principles. At a minimum, something that looks broadly like one of today's AI models would nee…

Yeah this original poster is only talking about the "theoretical" part of intelligence, and somehow completely forgetting about the "practical experimental" which is the only way to solidify and improve any theoretical things it comes up with
Post reply on HN