When I think of cancel culture, my primary thought is of private individuals facing meaningful harms (mainly economic) as a result of public outcry over the individuals (perfectly legal) speech or actions that signal the individual is "on the wrong team".
It's not a simple definition, but captures both why people are afraid of it, and why it is happening now (social media made it possible to make a private individual's speech and actions public, even if that individual wasn't a user of that platform).
Public figures losing speaking engagements or whatever is bad, but the targeting of private individuals in this manner is an escalation of political conflict that is very alarming.
Lastly, I will add that while this evil is not exclusively committed by the Left, there is absolutely an asymmetry. The Left has generally been far more likely to cancel people than the reverse. There are a number of possible causes (people on the left are more politically active, will be amplified by a left -leaning media industry, etc)