I am strongly in favor of the right to photograph anything and anybody in public places, but from a moral/ethical/legal point of view, "intent" is indeed everything.
You make it sound like it would be secondary and not really relevant, but for example, intent is the difference between murder and involuntary manslaughter.
In candid street photography, I think what people fear is exactly that: bad intent, a willingness of the photographer to exploit the images in order to show ridicule or maybe even obtain sexual gratification.
And it's also an easy way to do some virtue signalling. It's striking that in the article, it's not the people being photographed who object, but complete strangers to the scene, who know absolutely nothing about the specific circumstances of how the image was taken and whether or not consent was granted.
Of course, intent lies not just in the pressing of a shutter; one can also select stills from security footage with intent to ridicule. And indeed, as a general rule, access to security footage is limited and controlled, probably for that reason.