Let's imagine you and I play Roshambo - rock, paper, scissors. The game theory optimal solution here is easy - I randomly pick one. This means I will never score better than 50% (in the longrun), but I will also never score worse than 50%. Now if I actually tried to play and beat you then I could, in theory, win every single time if I somehow I just managed to really get into your head. But at the same time, I now open myself up to exploitation and you could end up beating me every single game.
So there are two ways we could approach this game. In the game theory optimal solution my goal is to minimize my worst possible outcome. In the exploitative solution my goal is to maximize my best possible outcome. This has an oddly direct analog to this conversation.
> "..who would’ve thought that putting short term self interest above everything else is not the best of ideas."
I fully agree with you. Elevating self interest to the highest assumption is an awful idea. But in practice it's the least awful of other even more awful ideas. By assuming the lowest common denominator (self interest > all) you create a scenario where the worst scenario, of people behaving in this way, is not catastrophic in any way. In fact it's wholly expected. You will probably never have a utopic outcome, but you'll also probably never have a catastrophic outcome.
Other systems have been trialed multiple times. These systems are phenomenal in theory. So long as every person agrees to cooperate and play their assigned role in a fair and equitable way we can have a utopia where no one ever need go without and everybody can live a comfortable life without fear of ever going without. The problem here is that these systems are extremely dependent upon good behavior. With good behavior, you get what some might call a utopia. But when bad behavior emerges, instead of a utopia you get starvation, oppression, and a general nightmare of existence.
---
Maybe one of the coolest things about capitalism though is that you're free to try any system you want, even within it! For instance the Garden of Eden [1] has been a phenomenally successful fully self sufficient independent commune. By contrast, if you'd like to try out a capitalist (or even some sort of communal system) within an enforced social economic system, you can't. And trying to do so would generally be met with a rather aggressive response from the government. Because once again, the sustenance of such systems require everybody play their role. If people just start doing whatever they want, the whole system would collapse. With a half dozen ideologically aligned people on a few acres of land, this works out phenomenally well. With millions of diverse people on millions or billions of acres of land, it tends to be somewhat less successful.
[1] - https://www.intothegardenofeden.com/