You need to ween yourself off your social proof addiction. The problem with vote counts is that two downvotes can start a cascade of reflex downvotes. Or at best, it won't get upvoted. When vote counts where active, this effect caused many instances where an excellent comment was found greyed out at the bottom of the page until a few smart HN folks with high karma voted it back up.
Maybe the solution is to only show positive vote counts. We were already stopped at -4, so not showing any count unless it's above 1 isn't that far off.