There's a great aphorism that applies here:
In theory, theory and practice are the same.So it is with regard to free speech, and it's a thorny wicket. In a free marketplace of ideas, people are able to hear all viewpoints and select the most rational ones. Unfortunately, there are studies that seem to suggest that the repetition of a falsehood reinforces the ideas contained therein for at least some of the population. The best I can find quickly are "How Warnings About False Claims Become Recommendations: Paradoxical Effects of Warnings on Beliefs of Older Consumers"[1] and "Metacognitive Experiences and the Intricacies of Setting People Straight: Implications for Debiasing and Public Information Campaigns"[2]. (Look at the birther movement. There's an insane number of people in the U.S. who believe something to be true that is not only nonsensical but has been absolutely proven false. To many of these people, the means by which things were proven false has only reinforced their insane belief—and the more they repeat it, the more often they have a chance to convince others of their insanity being true, even though objectively it isn't.)
We know that words can wound deeply; look at the concerns over "cyber-bullying" and the reports of preteen suicides over some of those cases. As a parallel, it's illegal in every country with the concept of freedom of speech to incite a riot (even though the rioters are the ones causing the damage). As wlievens said, there's often direct consequences to hateful speech[3] that are deeply tragic.
Like corin_, I'm divided on this matter. I grew up American and emigrated to Canada as an adult. Canada has hate speech laws that can be fairly described as "prior restraint" and wouldn't pass muster in America. Those offend me as an American. But the level of discourse in Canada is, I think, better for such prior restraint. Obviously, one must be cautious in how one applies such things, but the improved discourse pleases me as a Canadian.
This is a long way to say: no, I don't think that free speech is more important than these other issues just because it's free speech. It's a complex problem without an easy hard-line solution that can be applied effectively. Because of the effects of repetition, sometimes the best way to stop hateful or stupid speech is to squelch it. And sometimes, it's the right thing to do.
[1] pdf: http://www.cerog.org/lalondeCB/CB/2005_lalonde_seminar/how_w...
[2] pdf: http://sitemaker.umich.edu/norbert.schwarz/files/07_aep_schw...
[3] http://news.ycombinator.com/item?id=2358825