It totally matters. Just ask any Mac or Linux diehard. Personally, I use all three of the primary operating systems: Windows XP, Mac OSX and Linux (Ubuntu in this case). I have separate uses for each and can't imagine getting rid of any of them.
I'm curious what you use Windows for. The only things I can imagine myself using Windows for are gaming or testing in IE.
Well, to test in Internet Explorer of course! I'm a front-end engineer for Yahoo!, and unfortunately a stupidly high number of our users are IE users. There are options on other operating systems like Parallels and WINE, but nothing gets the job done like the real thing.