Earlier quoted context omitted.
Here's my question not to you specifically but the royal you of any bigTech employees reading this forum or similar: why do the employees not stand up at all hands meetings and raise this issue as a serious problem. Of course I know the answer in they just don't care and their personal paychecks are too high for them to risk becoming a squeaky wheel. However, it is obvious that management does not intend to fix this…
Because when you're dealing with a billion users, a one-in-a-million mistake still screws a thousand users, and everyone realizes that getting the rate of mistakes anywhere near that low is impossible. At the same time, you're dealing with actual bad actors that genuinely need to be banned, and there is a lot of this abuse. Mistakes will happen. You can't stop banning users. The bad actors will file appeals (in some…
> Abuse teams are (understandably) rather tight-lipped, and also tend to insist that telling the user what they did wrong would enable abusers to dodge the protections - this is something I don't understand (the abuser presumably knows what they did, while a user wrongfully accused does not), but it seems to be consistently said by abuse teams from many different companies.
The abuser will have done a lot of things. They don't know which one(s) got them caught. The more information they're revealed, the faster they can iterate to avoid getting caught next time.
But also, the real user doesn't actually benefit that much from knowing what got them banned. What are they going to do? Go back in time and not do it?
Now, a thing the user will greatly benefit from is knowing exactly what (if anything) they need to do to get their account back. That's where the focus on informing users should be. But since the abusers will also be told the same information, this needs to be something that legitimate users will find easy, but abusers or account hijackers will find hard to do at scale. So it can't just be "write a sob story about how you need the account back because it has photos of your dead grandmother". The abusers will be more competent at that than real users.
Typically it has to be spending a limited resource, or at least proving you have access to some limited resource and rate-limiting recovery actions by that resource. Some examples: non-VOIP phone numbers, proof of real world identity, social vouching by accounts in good standing.