Why do you think it might have been that the founding fathers were such advocates for freedom of speech? Do you think they were unaware that it would lead to the normalization of unpleasant things being said? These questions can be answered quite succinctly by asking one other question. In a nation without freedom of speech, who gets to determine what can and cannot be said?
The Treason Felony Act 1848 [1] is something that is still technically a part of the law in the United Kingdom even if not actively enforced. In 2001 The Guardian made a legal effort to finally have it officially removed from the books, and failed [2]. The act makes it illegal to call for the abolition of the monarchy within the United Kingdom, even "imagining" such is sufficient to conviction. The max penalty is life imprisonment -- quite progressive as it used to be death. This is a stark reminder of why and where the desire for free free speech came from. Imagine our past without freedom of speech. There undoubtedly would have been numerous states that would have made the mere advocacy for abolition illegal, others that would have made advocating for suffrage of various groups also illegal, and so on.
You may think it simple to demarcate a line between 'reasonable' and 'unreasonable' speech but it's not so easy and ultimately the individuals that get to decide as such are those with the most power in society. The lines end up being drawn as fairly and reasonably as our states draw the lines laying out their voting districts. Laws against free speech invariably end up being exploited to help entrench whatever political ideology happens to get a grip on power within a nation, as the UK laws past and present are a reminder of.
Such laws can even be used to enforced bigoted views. For instance in the UK in 2017 a 19 year old lady was arrested and convicted, forced to wear a ankle monitoring device, abide a curfew, etc for "sending a grossly offensive message by means of a public electronic communications network." Her crime? Quoting lyrics from a Snoop Dog song on Instagram. That conviction was overturned a couple of years later. In another case (again in the UK) a Christian preacher was arrested for stating that, while he was not homophobic, he believed that the Bible taught homosexuality was a sin. Again it was overturned, yet being arrested, let alone convicted, for such "offenses" is hardly a society any should thrive to emulate.
Ultimately, I think people only see things they disagree with as being affected. In reality once free speech goes it will also include some of your views you find in no way unacceptable. In Germany until 2018 it was illegal to publicly insult any head of state. It was revoked only when it became inconvenient to the powers that be. Following a comedian reading an obscene poem about Turkish president Erdoğan, Turkey demanded and lawfully received prosecution which ultimately led to the law's removal. Laws which make it illegal to "defame" the President of the German Federal Republic remain on the books.
[1] - https://en.wikipedia.org/wiki/Treason_Felony_Act_1848
[2] - https://www.theguardian.com/uk-news/2013/dec/13/calling-abol...