You're right: it's the
policy that's bad. However, facial recognition is what makes enforcing that policy possible, and society and the law haven't changed quickly enough to deal with the repercussions.
For instance, say 18 year old "Joe" gets caught stealing soda from a fast food place. The manager kicks him out and bans him from the store. Fast forward a decade, when Joe walks back into the store with his wife and kid to buy a quick meal.
Without facial recognition: Joe's grown up and matured. The manager was either replaced years ago, or doesn't remember the incident, or vaguely remembers but doesn't care about it anymore. Joe and his family pay, eat, and leave.
With facial recognition: the system notices that someone on its "do not allow" list has entered the store and summons police to deal with the trespasser.
When human judgment is involved, we rarely deal with absolutes. A lifetime ban isn't really for life. It's until both parties grow up and the situation cools down. A ban on a competitor's employees isn't absolute. Maybe you won't serve the owner, but if his dishwasher comes to your place on a date, you're not gonna hassle the kid. We're really, really bad at designing automated systems that handle nuance. It's way easier to write code like `if photo_hash in banned_people: ...`.