Let's see... 1.) You need a shit-load of people to play baseball anything like for-real. At minimum 15-20 people for two teams. Pretty tough to do as a kid, unless you're free to roam free all over town to gather a crew, which approximately 0% of kids have license/protection from CPS to do. 2.) You need a big enough area for a baseball diamond, where you don't have to worry about knocking out a window with a foul bal…
It solves some of your problems - you can see poor kids in India playing it in alleyways pretty much the same way rich kids in England do - and, I think, it's a bit easier to hit the ball.
Plus breaking for tea is part of the game...