I keep asking it, and nobody wants to answer because it doesn't fit within the paradigm of "making AGI"
What if intelligence requires agency ?
AGI is an engineering problem, not a model training problem
41–50 of 442 posts
Re: AGI is an engineering problem, not a model training problem
#42Earlier quoted context omitted.
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
> Would that really be a physics discovery?
No, it could be something that proves all of our fundamental mathematics wrong.
The GP just gave the more conservative option.
Re: AGI is an engineering problem, not a model training problem
#43Earlier quoted context omitted.
On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.
Do we have a specific enough definition of general intelligence that we can exclude all non-human animals?
Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?
Re: AGI is an engineering problem, not a model training problem
#44It's a research problem, a science problem. And then an engineering problem to industrialize it. How can we replicate intelligence if we don't even know how it emerges from our brains?
Re: AGI is an engineering problem, not a model training problem
#45The first premise of the argument is that LLMs are plateauing in capability and this is obvious from using them. It is not obvious to me.
Consensus on GPT-5 has been that it was underwhelming, and definitely a smaller jump than 3 to 4.
Re: AGI is an engineering problem, not a model training problem
#46[deleted]
Re: AGI is an engineering problem, not a model training problem
#47If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance. So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing…
Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with, and with what strength, to improve general intelligence? Surely it should focus on reliable academics, but which academics are reliable? How can we reliably ensure it studies topics that are "pure knowledge", and who does it choose to be, if we assume there is some theoretical point where it can autonomously outpace all of the world's best human-based research teams?
Re: AGI is an engineering problem, not a model training problem
#48Way out of touch. AGI is poorly defined and thus is a science "problem", and a very low priority one at that. No amount of engineering or model training is going to get us AGI until someone defines what properties are required and then researches what can be done to achieve them within our existing theories of computation which all computers being manufactured today are built upon.
It strikes me that until we fully understand human consciousness, we don't stand a chance of reaching AGI. Am I incorrect?
I think we can relax that a bit. We "just" need to understand some definition of cognition that satisfies our computational needs.
Natural language processing is definitely a huge step in that direction, but that's kinda all we've got for now with LLMs and they're still not that great.
Is there some lower level idea beneath linguistics from which natural language processing could emerge? Maybe. Would that lower level idea also produce some or all of the missing components that we need for "cognition"? Also a maybe.
What I can say for sure though is that all our hardware operates on this more linguistic understanding of what computation is. Machine code is strings of symbols. Is this not good enough? We don't know. That's where we're at today.
Re: AGI is an engineering problem, not a model training problem
#49We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think th…
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
It’s not really “what is the brain doing”; that path leads to “quantum mysticism”. What we lack is a good theoretical framework about complex emergence. More maths in this space please.
Intelligence is an emergent phenomenon; all the interesting stuff happens at the boundary of order and disorder but we don’t have good tools in this space.
Re: AGI is an engineering problem, not a model training problem
#50Earlier quoted context omitted.
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
That sounds like you’re describing AGI as being impractical to implement in an electronic computer, not impossible in principle.