There should be no group selection, it should be all or nothing. We also need to be damn careful of the recent attempt to brand speech some groups do not agree with as "weaponizing speech" which seems to be a new means to get the government to step in. With hate speech the government is still very limited on First Amendment grounds however by branding some speech as "weaponized" it implies violence and that does not have the same protection.
Frankly I don't mind the haters being out there, at least then I know who they are. Let them post, let them march, because in public venues we can at least assess when they are truly dangerous or not.
tl;dr censoring is fine when applied equally, be wary of anyone trying to change how speech is labeled as a means to get the government to do their censoring for them