That doesn't mean we should accept it from AI. We should fight the blind yielding to the facade of authority regardless of whether the decision was made by an AI or an insect landing on a teleprinter at the wrong time.
Innocent woman jailed after being misidentified using AI facial recognition
131–140 of 417 posts
Re: Innocent woman jailed after being misidentified using AI facial recognition
#132The cops cannot be held accountable because the laws basically give them immunity. The politicians cannot be held accountable beyond being tossed out at the next election, because the laws otherwise give them immunity. The people operating the system cannot be held accountable, because the systems are marketed as authoritative despite being black boxes and lacking in transparency; they trusted the system just as they were told to, and thus cannot be held accountable.
And so when every human in the chain cannot be held accountable for these things, and the law prevents victims from receiving apologies, let alone recourse, then the tool and its maker is the only thing we can hold accountable. By deflecting blame away from the tools ("it wasn't AI, it was facial recognition"; "the human had to sign off on it"; "humans made the arrest, not machines"), you're protecting quite literally the only possible entity that could still potentially be held accountable: the dipshits making these stupid things and marketing them as superior and authoritative when compared to humans.
You want accountability? Start holding capital to account, and this shit falls away real fucking fast. Don't get lost in technical nuance over very real human issues.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#133Earlier quoted context omitted.
I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
As soon as we start to see a pattern of shitty vibe-coded software actually harming people via defects etc. (see: therac-25), I would hope that the conversation is about structural change to mitigate risk in aggregate rather than just punitive consequences for the individual programmers who are "responsible". The latter would be a fantastically stupid response and would do little or nothing to reduce future harm.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#134It's not an AI error. It's a human error in mis-using AI in this way. Saying it's an AI error is like saying a hole in your drywall is a hammer error. Unfortunately we'll probably see a trend of people using AI and then blaming AI for cases where they mis-used AI in roles it's not good for or failed to review or monitor the AI.
It's both. It's good to acknowledge that AI is easy to misuse in this manner but it doesn't detract from the fact that the ultimate responsibility lies in those that should be verifying the tool output. There is far too little skepticism around the magic box that solves all problems which is causing issues like this. It's not the fault of the AI (as if it could be assigned liability) for being misused, but this kind…
Re: Innocent woman jailed after being misidentified using AI facial recognition
#135There's no way this isn't a slam dunk case to sue the piss out of the Fargo Police, probably the US Marshals and maybe other orgs. The woman in the surveillance phone clearly looks way younger, among the many other obvious signs this woman didn't do it. I hope she wrings at least several million dollars out of the government.
It literally doesn't matter -- you're focused on the wrong thing. She could be that woman's exact twin and it wouldn't matter. Spending six months in jail and losing your house, your car, and your dog with the flimsiest of evidence is ridiculous.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#136Earlier quoted context omitted.
It literally doesn't matter -- you're focused on the wrong thing. She could be that woman's exact twin and it wouldn't matter. Spending six months in jail and losing your house, your car, and your dog with the flimsiest of evidence is ridiculous.
A lawsuit is exactly what matters. They learn only the hard way, and no other way. If you want them to not be ridiculous, a lawsuit with large punitive damages is the only practical way to get there.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#137Earlier quoted context omitted.
As soon as we start to see a pattern of shitty vibe-coded software actually harming people via defects etc. (see: therac-25), I would hope that the conversation is about structural change to mitigate risk in aggregate rather than just punitive consequences for the individual programmers who are "responsible". The latter would be a fantastically stupid response and would do little or nothing to reduce future harm.
all accountability need not be punitive, we can certainly talk about systemic guardrails. What I find disbelief in, is someone saying the Chief of Police saying "We are not going to talk about that today?" is not the biggest scandal, but the AI is.
Who is this "someone"? OP's article and the discussion here are absolutely not neglecting the human factors and general institutional failure that made this possible. But it's also true that without these "AI" tools, it would never have happened.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#138Earlier quoted context omitted.
> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…
I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
We're only getting warmed up. There are programmers on HN that will take the output of their favorite AI, paste it and run it. And we're supposed to be the ones that know better.
What do you think an ordinary person is going to do in the presence of something that they can not relate to anything else except for an oracle, assuming they know the term? You put anything in there and out pops this extremely polished looking document, something that looks better than whatever you would put together yourself with a bunch of information on it that contains all kinds of juicy language geared up to make you believe the payload. And it does that in a split second. It's absolutely magical to those in the know, let alone to those that are not.
They're going to fall for it, without a second thought.
And they're going to draw consequences from it that you thought could use a little skepticism. Too late now.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#139Earlier quoted context omitted.
> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…
I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
Unfortunately, a lot of people are certain it won't happen to them, and it has been practically impossible to establish any kind of accountability. It has only gotten worse since 2020.
Re: Innocent woman jailed after being misidentified using AI facial recognition
#140Earlier quoted context omitted.
I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.
It's called qualified immunity. Many support its repeal. I hope you join them, and convey the same to your local representatives and candidates. Until it is reformed few if any officers or administrators of criminal justice in the United States will ever feel any type of accountability. Short of video evidence of blatant gun to the back of the head style homicide qualified immunity means most law enforcement official…
And frequently not even then.