> It implies that whether we should allow totalitarian measures merely depends on a judgement of case by case merit
To some extent, all effective government is necessarily conducted based on case by case merit. Even what we think of as fundamental human rights can never truly be sacrosanct, for the simple reason that sometimes they conflict. One obvious recurring example in recent years has been the tension between security (right to life etc.) and secrecy (right to private life).
Should we protect some values more than others? Sure. Should some principles be considered too important for the administration of the day to legislate them away without going to the people explicitly first? Absolutely. Can you ever codify that in a reasonable way without allowing some degree of case-by-case judgement? I doubt it.
> So what if the authorities decide one day there is no good side?
Then you move onto the next of the four boxes of liberty. If lots of people agree with you, then one way or another, either the authorities will back down or they will be removed from power.
The more I think about these issues, the more I conclude that the law can only ever be an instrument of a civil society. If you reach the point where one significant group is not behaving in a civilised fashion and the rest cannot persuade them to do so through reason, then ultimately it rarely matters what the law says until that disagreement has been settled by force. On a small scale, that is called policing. On a large scale, it is called civil war.
If you accept that premise, then it only ever makes sense to balance the law favourably for the civil case, on the presumption that the legal system is operated in the interests of justice. If it is not in a few isolated cases, you treat the deviants as criminals themselves and you provide an appeals mechanism for those who have been treated unfairly. If it is not on a larger scale, then the law doesn't really matter anyway, because your only recourse is to bring down the system and build a new one. You can do that by voting out the government, through civil disobedience that puts unbearable pressure on the system, or on a more fundamental level as we've seen in several former dictatorships over the past year.