Maybe the harm will be realized in the long run, when you want to watch something that is no longer available?
For example, the Netflix library changes every now and then (at least that I've been able to detect), and there's more than one show that I can no longer watch anymore that I wish I could.
That to me is harmful because I don't really know when this happens until I search for it and it's no longer there. So I end up buying DVDs of shows/movies I really would like to re-watch in the future just so I can avoid Netflix's listing changes.
I guess the argument is that if there were no DRM I could download those shows I like the most and watch them at anytime, regardless of whether Netflix wants me to or not.
Not sure about the reasons for others to consider DRM harmful (or not), but this one to me is really important and would love to have a DRM-free alternative besides pirating (due to inconvenience mainly, and due to general concern for downloading stuff that might not be what it supposedly should be).
However this looks like wishful thinking at best, as time goes by and DRM is pushed further.