I think a key problem is our understanding of the quality of an ML system is tied to a task. Our mechanism of training is tied to a loss, or some optimization problem. The design, training, and evaluation of these systems is dependent on an externally provided definition of "correct".
But this seems structurally different from how we or even less intelligent animals operate. DALL-E may make "better" art than most humans -- but it does so in response to a human-provided prompt, according to a system trained on human produced or selected images, improving on an externally-provided loss. Whereas a human artist, even if mediocre, is directed by their own interests and judges according to their own aesthetics. Even if some of their outputs are sometimes comparable, they're not really engaged in the same activity.
Methodologically, how do we create agents that aren't just good at several tasks, but make up their own tasks, "play", develop changing preferences for different activities (I think this is more than just "exploration"), etc? Even a dog sometimes wants to play with a toy, sometimes wants to run and chase, sometimes wants to be warm inside. We don't "score" how well it plays with a toy, but we take its desire to play as a signs of greater intelligence than, e.g. a pet iguana which doesn't seem to have such a desire.
Further, how do we create agents that can learn without ever seriously failing? RL systems have many episodes, some of which can end very badly (e.g. your simulated runner falls off the world) and they get to learn from this. We die exactly once, and we don't get to learn from it. Note, learning from others in a social context may be part of it, but non-social animals also can learn to avoid many kinds of serious harm without first experiencing it.
I don't mean to overly discount the current methods -- they're achieving amazing results. But I think even an optimist should be open to the possibility / opportunity that perhaps the current techniques will get us 80% of the way there, but that there are still some important tricks to be discovered.