It's case by case. For books I like to go shopping/browsing in physical stores, which is just more fun. And no waiting for shipping.
For video games/movies I follow RSS feeds of some deal monitoring sites and buy wherever I can get a good deal. Sometimes it's Target, or Best Buy, sometimes it's a random online retailer that's legit, but I wouldn't have otherwise considered. It's a competitive business and the same deals pop up in more than one place.
I used to get eye drops and vitamins on Amazon, the everyday price at CVS is higher than Amazon but they regularly go on sale at CVS and I do just as well by just timing the purchase. I'm in CVS all the time anyway.
I haven't made a major purchase like a TV this year but I'm sure I'd do fine if I waited for a specific sale or the general holiday season deals.
And sometimes I just don't buy something at all. I started out not wanting to buy from Amazon because of how they treat their employees and business partners. In the process of rethinking that I also realized I don't need to be buying so much stuff anyway. I used to throw in items so I'd qualify for free shipping on $25 (now $35) orders. Now I buy fewer things, only when I'm sure I'll really use them. Instead of building up a backlog of books I'll get to someday, I buy one when I'm ready to start reading it.
I bought a pair of $8 earphones from Best Buy and got free shipping for it alone, at Amazon it would have been $6 but shipping would have been extra unless I bought more. The cost of Prime amortized to my number of purchases would not be a good shipping rate. (I'm not interested in the other Prime benefits.)
I've found that dropping loyalty to any one retailer gives me the best overall experience and/or the best deals.