That only makes sense in case of "me too" comments. You have 4 options of response to somebodies comment:
1. Agreeing while providing some new information. So you write a comment (actually you'll probably upvote in this case as well, but whatever).
2. Agreeing without providing any new information. Like, "yeah it's completely right what this guy just wrote". That comment wouldn't be very useful indeed, so that's why upvoting exists. And that's totally normal option, because it's very much possible that some guy just "nailed it" and thus you don't have much to add, yet you want to show your approvement somehow.
3. Disagreeing providing new information. Like, "no, your statement is false, because that and that, here you have logical mistake and there you just got facts messed up: here some link for you to verify that". Obviously, that would be a comment. And that's what constructive conversation is made of.
4. Disagreeing without providing any new information, so you have downvo… but hey, wait, what was that? You can agree without providing any new information, but disagreement is meant to have some reasons for it. So if I disagree I'd better clarify why I disagree or just remain silent completely. Basically downvoting here just means "I don't like your comment" and it's quite reasonable that one shouldn't be able to do that unless he has something more to say. There's just no sense in such a thing as "disagreeing without explanation", unless that guy you disagree with isn't an obvious troll, and, honestly, you can never know if somebody is "an obvious troll" — it's quite likely that you just don't understand his reasoning, so if you don't want to continue discussion you'd better just ignore him.