Yeah, it's definitely tough to try out speakers.
I try to be a little aware of speakers in friend's houses and stores to see what things sound like, but I don't purposely audition them. Although very few of my friends have decent speakers.
It sounds like you know a bit about what you want. My use case is also music. I used to have a surround setup, and it just wasn't worth it. And I agree with you on the subwoofer thing -- I had one and addition to the physical downsides, it felt unbalanced.
In my case I realized I like the imaging of Bose speakers, but they are cheaply made and seem a bit scammy and overpriced. Bose isn't worthless -- they do sound DIFFERENT, on purpose -- but they are flawed. There is real engineering there.
I asked Don Lindich about small speakers with good imaging that are not Bose, and he recommended Mirage Omnisats. It basically gives you a spacious sound like Bose, but they are higher quality in other respects, and I think cheaper.
I used to have Mirage Omnisats with a sub. Now I have Ohm Walsh 1000's:
http://ohmspeaker.com/speakers/mains/walsh-tall/
Admittedly this was a blind Internet buy, but they have a long (90 day) return policy, and I trusted Don's recommendation.
They sound fantastic, and people who have come to my apartment love them, and start spontaneously dancing when they hear them. Relatively speaking, they're not super expensive, but most people haven't heard speakers this good in a living room.
I also appreciate a small company that has been around for decades doing the SAME thing -- really refining it. They believe in what they're doing. You can't derive a speaker design from first principles, so you need to have a long history, with lots of trial and error.
Perhaps this doesn't really answer your question, but that's how I got to a set I'm very happy with.
As for the physical aspects, good speakers require fickle and custom manufacturing techniques. And, all things being equal, bigger speakers sound better, and bigger usually means more expensive.
Amplifiers and cables are mostly made of commodity parts as far as I can tell. There is also just more of a potential variance in quality of design with speakers, so I think they require more expertise to design than amplifiers. Bad speakers are really bad, but it's hard to buy an amp that's unlistenable.
So I guess if like me you are short on friends who like quality audio, I would visit stores, ask like-minded people on the Internet, and then make liberal use of return policies. There is also just a bunch of trial and error, and there's a reason why audiophiles tend to own many pairs of speakers that they don't need :)