It seems to me that the best way to think of Android is to compare it to the world of Linux distros. (I believe Dan Benjamin of 5by5 pointed this out on one of his podcasts.) Yes, there's Linux, but there's no one Linux. There's Android, but there's no one Android. So while Android is technically an operating system, it's really an open platform on which to build mobile devices on top of. So the Android device I buy…
With Android, you have to. Some devices will not be capable to run your app, in some others your app will not even fit the screen... And, of course, because of different drivers, hardware and OS versions your app will not absolutely work on some phones because of some weird error. It's nowhere near Linux development.
I don't know if fragmentation is good or bad for the users, but for developers it's an infinite source of headaches. From my experience (I develop a Twitter client for WP7), even with a system with much less fragmentation than Android, every phone has unique features which can make your app crash for no reason, and you can only wait to be randomly corrected or get that specific model and debugging the app.