> The right to free speech does not protect your right to commit crimes or [...]
If hate speech were to be made a crime, your exact same argument "the right to free speech does not protect your right to commit crimes" could and would be used as an argument why the First Amendment supposedly does not apply.
> It neither helps cause physical damage nor costs the victims money. You cannot, in the US, ban the expression of a thought just because you think somewhere down the line it could result in damage to someone.
That is exactly what your example of planning to rob a bank is. The planning to rob a bank does not cause any damage. The planning to rob a bank does not cause any victims any money. Actually executing that plan does that, but the planning does not guarantee that the execution will happen. Yet if I were to say "You cannot, in the US, ban it just because you think somewhere down the line it could result in damage to someone" you would rightly dismiss it as nonsense.
> Hate speech is mostly just hurting peoples feelings.
We may disagree on the "mostly", but even you are acknowledging that it is not limited to that, so let's focus on cases that can't be considered that.
> Libel is a tort because you lie (important) to damage someone's reputation and this could cost them money.
Okay, now consider for instance well-known conspiracy theories that the world is controlled by (insert group here). They are lies, they damage the reputation of many if not most members of the group, and this could cost members of the group money. They are not libellous if not said about specific people, they are generally considered hate speech if based on any of a specific set of protected characteristics, but despite the exact same reasoning applying to both, libel would not be considered protected speech, and, as far as I understand, this particular form of hate speech would be.
It is also easy to think up scenarios where supposedly constitutionally protected hate speech "helps" cause physical damage: if without the hate speech, a group of people would be left alone, but with the hate speech, other people become convinced that it is morally defensible or even necessary to commit crimes against those people.
I can see no distinction in the constitution to justify this. I understand that this is how US courts interpret the constitution, but everything I can see tells me this is an arbitrary distinction drawn up by US courts with no actual basis in the constitution.