So let's consider the alternative - forcing people to comment on downvoting.
Let's say I see a post that is clearly not contributing to the discussion (on my personal opinion of course, not that I have any other one :) or is too misguided to even argue with, or lacks any redeeming merit, etc. Now, I want less of this on HN, so I am going to downvote it.
Now, let's say I'm forced to comment on it. I can't really say anything useful about it, and maybe even if I could I don't want to bother - if it was worthy discussion-wide, I wouldn't want to -1 it in the first place - so I'd say something like "this is just stupid". Or "-1 for not presenting a sane argument" or something like that.
Now, what happens when the comment author sees it? He comments "no, your face is stupid!" of course. And so we get a whole mess we wanted to avoid.
And now people see this whole mess and want to bury it so it doesn't stink up their nice discussion. But they are forced to comment too, so instead of removing the mess they are making it worse and make people hate each other for no reason but disagreeing on who started this mess.
Does not look like an improvement to me.