If people would just stop caring about meaningless numbers displayed next to your username, the world would be a better place. Specifically, the world of community-moderated sites wouldn't have to put up with crap submissions that exist solely to gather up-votes.
Sometimes I read a troll-like post, and check out the user that wrote it. If they have 3000 karma, it is likely that I am missing their point, and need to think some more. If they have -10 karma, then they are probably just a troll.
I know it's "bad" to judge someone by a number... but we do it because it is usually helpful.