You can still live like what's described in this article. Get yourself a dumbphone and a paper atlas, only pay with cash, avoid loyalty cards, read paper books, newspapers, etc Now and then I do that, just to switch off from our hyper-connected world. Switching off is the new peace of mind.
The thing about technology is that even if you don't change everyone else will. Sure, it might still be technically legal to ride a horse down the street. But soon enough, people started putting in multi-lane highways. Also, stores took out their hitching posts. Then we started designing cities around the car, so what used to be a mile away is now 10. And half of that distance is consumed by parking lots.
I haven't ridden a horse since I was 3 or so, but I rode a steam train through town last Sunday, which would be period accurate with your horse, and people smiled. They waved and took photos and thought it was really cool. Which it was.
It isn't practical, as you say, but it is pretty awesome. And because of that people will accommodate you.