>I ponder about it.
I did that too.
When it comes to genetics, you have to figure that underlying genetics set the baseline for the ability to respond to both culture & environment.
OTOH, genetics itself is quite fluid and adaptable on its own, capable of responding to both culture and environment over (sometimes unimaginable numbers of) generations not near the scale of a single individual's lifetime.
Making the breadth of what can be accomplished for any one person quite limited compared to what extremes may be accomplished over millennia under conditions that favor the desirable characteristics.
Whether the strongest influence is culture, environment, or both, has got to be an overwhelmingly complex set of variables.
Sometimes the culture is more stable over generations than the environment, or vice versa, sometimes neither one. Sometimes the genetics are more stable too, sometimes not.
Plus you can only imagine how bizarre some prehistoric cultures and/or environments were, and how some of the flourishers or surviviors deviated from others. Especially considering some environments and cultures were more widespread than others, or at the other extreme, relatively to completely isolated for more time than anything in recordable history. With a difference amounting to many orders of magnitude over all variables.
What are the odds that only one culture or environment selected for the hallmark learning, grasping, solving behavior that others are just not going to reach in a single lifetime? And from the variation that can be seen today, what are the odds that some other culture(s) did not deselect for the exact same characteristics?
If there was some lost culture of isolated super-intellectuals in the far prehistoric past, they were all just average people among themselves anyway.
But anything that far back can sure have an outsized outcome if the unconventional course is followed from such an early start, and the deviation from mainstream widens accordingly.
Regardless, I believe it is accepted that nobody is using but a small fraction of their brain constructively, and there are so many different kinds of talent. There might even be a common trend to using only a small portion of a single side of the brain more than anything else. So what.
So just about anybody should be able to mostly double their mental ability in a chosen area, and have plenty of room to spare, still utilizing only a small amount of what they were given to work with.
It's not going to happen unless you put your mind to it.