Earlier quoted context omitted.
Let's rephrase it as "when they act in furtherance to committing a crime", then (i.e. the definition of conspiracy sans requiring multiple people). And yes, it does mean that you won't catch all such people, only most. It's the case of diminishing returns - you have to radically increase surveillance and police state intrusiveness to go after fewer and fewer people, while also affecting more and more people who aren'…
That's all true, but only considers state-criminal relations. I urge reconsidering this from the point of view of political conflict rather than simple criminology.
If you mean, when one should start worrying about one's political opponents, the answer is, and always has been - as soon as their support starts growing for any reason. In that sense, the time for anybody on the left to worry about right-wing extremist politics has been many years ago, as was the time to translate it into action (i.e. vote strategically to defeat them as the highest priority).