Earlier quoted context omitted.
I don't think you're wrong that AGI will have (something like) those components in its inception. I just disagree that we have to intentionally create them -- I think there are a lot of places we can experimentally bruteforce the implementation instead of having a solid theoretical understanding if our only goal is creating an AGI. I don't think we need any theoretical breakthroughs, per se, but rather, a lot of comp…
I think you are right that we could, in theory experimentally brute force intelligence. But if we try and do that, we are basically trying to simulate an entire universe and wait for intelligence. But the amount of computing power that that would require is beyond even what we can imagine. And even if we could find some ways of simplifying it, it still seems like a waste of our intelligence. Like when it got cold for…
Humans already accidentally invented AI at least a few times, sort of. To talk meaningfully about this topic, I think it's better to step away from the topic of "intelligence" and instead speak about "reasoning ability". It's still fuzzy, but seems to sidestep a lot of the "can submarines swim" arguments.
Things like societies, governments, corporations, economies, etc all show similar features to machine learning algorithms and reasoning animals in their ability to dynamically problem solve -- the main difference is that they run on human computers instead of electronic computers. (Or photonic, quantum, etc computers.) Well, in modern times, they're actually sort of cyborgs, but that's a side issue.
The quest for an AGI is more about translating the notion of a corporation (or government, etc) meaningfully into fully electronic computer terms than it is in replicating the human mind. We know enough about the supra-structure of the corporation to meaningfully bound the search, and it's much more prudent to spend a decade on bruteforcing each component of a corporation than it is to navel gaze about reinventing minds. The truth is that human brains (or animal brains) aren't the only general intelligence that we know -- so reinventing them isn't the only path to AGI, though it's a worthwhile project in its own right.
I just worry that many of the meaningful contributions of the people working on reinventing minds will be lost on the birth of the actual first AGI through corporate cyberization. (And that many of us will fail to recognize corporate cyberization for what it is -- an AGI. Any (successful) corporation which can act fully by algorithm must be a general intelligence.)