And yet the elected leaders of one party have made it their crusade to deny access to abortion and make it harder to obtain contraception, believe that being gay or trans is a disease that can be fixed by therapy and that they should be excluded from the military, try their hardest to put roadblocks in front of several minority groups who just want to vote (and have flat-out admitted that the reason is because when more people vote, their party loses), strategically redraw voting districts to ensure they remain in power (often against the majority or significant minority), believe that women reporting sexual harassment and abuse are just looking for attention, think that systemic sexism and racism doesn't exist, hypocritically fight against immigration that has enriched them and enriches the country, and worse. I could go on, but we'd be here all day.
Note that I didn't say anything about white supremacists or nazis above. This is the party platform, and people who support that party either support -- or at least aren't strongly enough against -- these policies. And I really can't tolerate the attitude of "I want lower taxes and less regulation and smaller government so much that I'm going to vote in people who want to deny certain people their rights". I don't think these people are nazis, but I am continually disappointed that they make choices to enrich themselves at the expense of others. It doesn't have to be a zero-sum game, and many voters end up misunderstanding the effects of some policies to the point that they're actually voting against their own interests anyway.
I won't claim that the other party is all rainbows and virtue, or that they don't pull dirty tricks when they get the chance, but at least they generally don't try to take people's civil (and voting!) rights away or advance laws based on religious beliefs that discriminate against minority groups. It's telling that nearly all of the things I dislike about the Democrat party platform (expansion of surveillance state, willingness to invade foreign nations, lack of respect for due process, war on drugs, tough on crime) just happens to be shared across both parties, and often the Republican party takes a much more extreme stance on those issues.
I will also absolutely agree with you that cancel culture has gotten out of hand to the point of having a chilling effect on discussion that could be productive, critical, and worthwhile. And many people do make the awful, lazy, polarizing associations that you describe ("white supremacists are far-right so all conservatives must be white supremacists"). It's lame and I hate it. But let's not pretend that means that there aren't serious issues with the right's attitude toward civil rights, freedom from religion, equity and equality, social justice, criminal justice, and other things. If you vote Republican, you're voting for those things, even if you don't actively support them. We certainly shouldn't ostracize or alienate people who we don't agree with, or refuse to talk to them, but that doesn't mean we shouldn't call out people for supporting reprehensible policy.