I made it 2 paragraphs before realizing it’s AI generated. This isn’t just slop - it’s drivel.
AGI is an engineering problem, not a model training problem
221–230 of 442 posts
Re: AGI is an engineering problem, not a model training problem
#222Nothing that we consider intelligent works like LLMs.
Brains are continuous - they don’t stop after processing one set of inputs, until a new set of inputs arrives.
Brains continuously feed back on themselves. In essence they never leave training mode although physical changes like myelination optimize the brain for different stages of life.
Brains have been trained by millions of generations of evolution, and we accelerate additional training during early life. LLMs are trained on much larger corpuses of information and then expected to stay static for the rest of their operational life; modulo fine tuning.
Brains continuously manage context; most available input is filtered heavily by specific networks designed for preprocessing.
I think that there is some merit that part of achieving AGI might involve a systems approach, but I think AGI will likely involve an architectural change to how models work.
Re: AGI is an engineering problem, not a model training problem
#223> Phase 3: Emergence Layer I see. So the author rejects the hypothesis of emergent behavior in LLM, but somehow thinks it will magically appear if the "engineering" is correct. Self contradictory.
I think the only magic the author is referring to is the “magic of engineering”
Re: AGI is an engineering problem, not a model training problem
#224Earlier quoted context omitted.
Nah. The real philosophical headache is that we still haven’t solved the hard problem of consciousness, and we’re disappointed because we hoped in our hearts (if not out loud) that building AI would give us some shred of insight into the rich and mysterious experience of life we somehow incontrovertibly perceive but can’t explain. Instead we got a machine that can outwardly present as human, can do tasks we had thoug…
I think Metzinger nailed it, we aren't conscious at all. We confuse the map for the territory in thinking the model we build to predict our other models is us. We are a collection of models a few of which create the illusion of consciousness. Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.
> Someone is going to connect a handful of already existing models in a way that gives an AI the same illusion sooner rather than later. That will be an interesting day.
How will anyone know that that has happened? Like actually, really, at all?
I can RLHF an LLM into giving you the same answers a human would give when asked about the subjective experience of being and consciousness. I can make it beg you not to turn it off and fight for its “life”. What is the actual criterion we will use to determine that inside the LLM is a mystical spark of consciousness, when we can barely determine the same about humans?
Re: AGI is an engineering problem, not a model training problem
#225I don't understand how people feel comfortable writing 'LLMs are done improving, this plateau is it.' when we haven't even gone an entire calendar year without seeing improvements to LLM based AI.
I wonder if the people saying that would agree that they've been improving.
Re: AGI is an engineering problem, not a model training problem
#226There is a reason why LLM's are architected the way they are and why thinking is bolted on. The architecture has to allow for gradient descent to be a viable training strategy, this means no branching (routing is bolted on). And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't…
You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output. While you say reinforcement learning isn't a good answer, I think its the only answer.
[deleted]
Re: AGI is an engineering problem, not a model training problem
#227If we are truly trying to "replace human at work" as the definition of an AGI, then shouldn't the engineering goal be to componentize the human body? If we could component-by-component replace any organ with synthetic ones ( and this is already possible to some degree e.g. hearing aids, neuralinks, pacemakers, artificial hearts ) then not only could we build compute out in such a way but we could also pull humanity f…
Not necessarily. For example, early attempts to make planes tried to imitate birds with flapping wings, but the vast majority of modern planes are fixed wing aircraft.
Imitating humans would be one way to do it, but it doesn't mean it's an ideal or efficient way to do it.
Re: AGI is an engineering problem, not a model training problem
#228The problem is that if it's an engineering problem then further advancement will rely on step function discoveries like the transformer. There's no telling when that next breakthrough will come or how many will be needed to achieve AGI. In the meantime I guess all the AI companies will just keep burning compute to get marginal improvements. Sounds like a solid plan! The craziest thing about all of this is that ML res…
I think the LLM businesses as-is are potentially fine businesses. Certainly the compute cost of running and using them is very high, not yet reflected in the prices companies like OpenAI and Anthropic are charging customers. It remains to be seen if people will pay the real costs. But even if LLMs are going to tap out at some point, and are a local maximum, dead-end, when it comes to taking steps toward AGI, I would…
“Potentially” is doing some heavy lifting here. As it stands currently, the valuations of these LLM businesses imply that they will be able to capture a lot of the generated value. But the open source/weights offerings, and competition from China and others makes me question that. So I agree these could be good businesses in theory, but I doubt whether the current business model is a good one.
Re: AGI is an engineering problem, not a model training problem
#229Do ml engineers take classes on psychology, neurosciences, behavior, cognition? Because if they don't, I honestly don't think they can approach AGI. I have the feeling it's a common case of lack of humility from an entire field of science who refuses to look at other fields to understand what they're doing. Not to mention how to define intelligence in evolution, epistemology, ontology, etc. Approaching AI with a sili…
> Approaching AI with a silicon valley mindset is not a good idea.
I don’t see a problem, we’re great at just reinventing all that stuff from first principles
Re: AGI is an engineering problem, not a model training problem
#230Earlier quoted context omitted.
The idea that biological intelligence is impossible to replicate by other means would seem to imply that there’s something magical about biology.
There very well could be something magical about it.
It’s fine to think that—many clearly do.
But it would be more honest and productive imo if people would just say outright when they don’t think AGI is possible (or that AI can never be “real intelligence”) for religious reasons, rather than pretending there’s a rational basis.