Earlier quoted context omitted.
Hm... I always thought it has to do with the comments that get slowly grayed out? E.g., for me that was basically a "downvote".
There is a downvote button, but you need some minimum karma for it to be available to you. I can understand why it might be confusing if there's no downvote arrow visible!
Yeah, that some kind of UX pardigm - don't hide the control. Disable it, and explain the user why he cannot use it at the moment.
On the other hand, though, I can now simply collapse huge branches without downvoting anyone, so thanks for the clarification! :)