Inefficient market means someone is getting a bad price. That someone is going to be a retail/naive investor - Bill and Nancy who want to invest in Apple with part of their salary.
Sophisticated investors would never be on the other side of the trade. Add to that bigger spreads and unsophisticated market participants get fleeced. As they wouldn't want to get fleeced they would hire an intermediary to do trades for them - a stock broker who again charges commission.
We already had that: spreads were bigger, prices were less efficient. The main victims were retail/naive investors and the main beneficiaries would be big institutions, brokers and sophisticated investors.
Current system benefits:
-retail/unsophisticated investors
-vanishingly small group of the best/fastest/most sophisticated traders
Who lost:
-big institutions, banks, stock brokers
-investing wannabes who no longer get access to easy money from retail/naive buyers
You seem to assume there is a pool of sharks that prey on pure souls without fast connection. It's not the case at all. Most HFTs lose money without exchange incentives which are provided because exchanges understand liquidity and low spreads are good for other participants. The pool of people who make money out of spreads/inefficient prices is way lower than it used to be as well.
The best possible world would have efficient prices every nano-second coming from god himself. Having a group of very sophisticated market makers is the second best we can have - at least if you care about normal investors who don't have resources to process all the information themselves.