On the contrary, the grocers' prices are the very heart of the matter. You see, the Ferrari neighbor may be HFT, but so are the grocers. And every grocer wants to sell their goods at a newly-higher price if they get wind of a big buyer, because it will cost them more to restock if you buy everything. So they will naturally raise their prices. If you buy from the grocers one at a time, expect them to call the other stores (ok it's a chain grocery) to alert them. You will pay a lot for your last milk purchase, unless you stop buying because the asking price has gone too high. If you instead have milk trucks make synchronized purchases across town, you will pay the same price everywhere, and all of the grocers will be upset once they realize what happened. If you do the synchronized milk truck plan enough, the grocers may permanently raise their milk prices at the expense of losing a few other price-conscience buyers.
In reality, this occurs i the stock market because the "national market" consists of something like 15 different exchanges in a big distributed system, so there are race conditions. HFT market makers trade on all of the exchanges, and adjust their prices based on trading demand seen from other exchanges. They spend a lot of money on fast networks between the exchanges, so that they can be/beat the proverbial ferarri. But synchronized trades a la tgemilk trucks or Katsayuma's "Thor" cannot be raced against.
IEX is intended to partially commoditize the Thor approach: take the advantage away from the grocers (HFT, market makers) and give it to the big milk buyers (institutions like hedge funds and pension funds). Retail trading is not affected one way or another.