Live data from Hacker News

Innocent woman jailed after being misidentified using AI facial recognition

grandforksherald.com

161–170 of 417 posts

Re: Innocent woman jailed after being misidentified using AI facial recognition

#161
post #2

> facial recognition showed she was the main suspect in what Fargo police called an organized bank fraud case. > Her bank records showed she was more than 1,200 miles away, at home in Tennessee at the same time police claimed she was in Fargo committing fraud. > Unable to pay her bills from jail, she lost her home, her car and even her dog

The Fargo DA should be fired, at a minimum.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#162
post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

> Instead of scapegoating an AI bogeyman

One big reason for AI adoption everywhere is that you can use it as a scapegoat

Re: Innocent woman jailed after being misidentified using AI facial recognition

#163

I really, really need folks to understand that deflecting blame away from the tool and trying to hold the human accountable feeds right into the marketing playbook of these companies in the first place. The cops cannot be held accountable because the laws basically give them immunity. The politicians cannot be held accountable beyond being tossed out at the next election, because the laws otherwise give them immunity…

Strongly agree here. This is an extremely predictable outcome of selling AI facial recognition software to American police forces.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#164
post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

This particular "AI bogeyman" isn't just AI; it's cops with AI and in particular cops with facial recognition tools, dragnet LPR surveillance tools, and all this other new technology that essentially picks somebody's name out of a hat to have their life temporarily (or [semi-]permanently) ruined by shithead cops who won't ever face any real accountability. This keeps happening, and the reason it keeps happening is th…

It's not just the shithead cops, it's the voters. All the "Blue Lives Matter", "thin blue line", "back the blue" propaganda works towards giving police infinite powers with zero accountability. This is what voters want and they've said so loudly over and over again.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#165
post #91

We are rapidly becoming a world where every person is one inscrutable LLM decision from having their life ruined with no recourse. This type of incident isn't new and is only going to get worse. The problem is our governments are doing absolutely nothing about it. I'll give two examples: 1. Hertz implemented a system where they falsely reported cars as being stolen. People were arrested and went to jail for rental ca…

> The problem is our governments are doing absolutely nothing about it

Huh. I thought they were actively accelerating the process. Hoping you are right and I am wrong.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#166
post #143

Earlier quoted context omitted.

I don't know what you're responding to, but I don't think it's my comment. Qualified immunity protects individuals, not departments, from liability. The particular thread (in this thread) that I was responding to: >> I hope she wrings at least several million dollars out of the government. > With all the lovely qualified immunity doctrine? That's wishful thinking. I was responding to the claim that qualified immunity…

The GP seems to be suggesting that there's no recourse at all, usually. You might bring suit against a police department or LE agency, but you'll fail to find any relief there. True that qualified immunity only protects individuals, but there's a raft of other things that makes it hard to get a judgement against a police department, too. I think there's probably one major exception: civil rights violation investigati…

> You might bring suit against a police department or LE agency, but you'll fail to find any relief there.

I don't know if I'd go so far to say she won't find any relief, but it probably still could be a pretty tough Monell claim against the department (although it's hard to tell from the sparse details in the article):

"[A] local government may not be sued under [42 U.S.C.] § 1983 for an injury inflicted solely by its employees or agents. Instead, it is when execution of a government's policy or custom, whether made by its lawmakers or by those whose edicts or acts may fairly be said to represent official policy, inflicts the injury that the government, as an entity, is responsible under § 1983." [1]

I could see a problem if there was a policy/custom of relying on AI facial recognition alone without any other corroborating evidence (would be a really stupid practice, but I'm sure stupider things have become part of a police department's systemic practices). Or if there was a failure to sufficiently train detectives about the erroneous tendencies of this technology. Maybe the needlessly prolonged detention without bail could be an issue if there was a lack of adequate protocols to expedite in a reasonable amount of time.

Either way, still seems hard to say this a slam dunk case for her, unfortunately. But also seems too risky for the city of Fargo to not settle, at least nominally.

[1] Monell v. Department of Soc. Svcs., 436 U.S. 658 (1978), https://supreme.justia.com/cases/federal/us/436/658/

Re: Innocent woman jailed after being misidentified using AI facial recognition

#167
post #86
post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…

It's not even just incompetence, but malice. "AI says so" is going to be the perfect catch-all excuse for literally everything anyone might want to do that they shouldn't. You know how techbros love to excuse every horrifying outcome of their torment nexi with "don't blame me, the algorithm did it"? It's going to be like that, but now everyone can do it.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#168
post #134
post #72

Earlier quoted context omitted.

It's both. It's good to acknowledge that AI is easy to misuse in this manner but it doesn't detract from the fact that the ultimate responsibility lies in those that should be verifying the tool output. There is far too little skepticism around the magic box that solves all problems which is causing issues like this. It's not the fault of the AI (as if it could be assigned liability) for being misused, but this kind…

I worry that blaming AI at all actually incentivizes humans to offload things to AI that should not be offloaded, since it lets them escape blame.

That is a huge danger. Legally speaking it's not an issue since misusing a tool doesn't relieve liability (in most circumstances - all the trivial ones at least)... but that's a more significant political issue as evidenced by the Anthropic vs. DoD interactions since the DoD's actions are largely immune to oversight by the justice system.

Of course, that depends on sane non-politicized courts which you may rightfully doubt exist right now - but assuming the system works anywhere near as designed outsourcing a decision to AI wouldn't change liability.

For DC fans Harvey Dent would similarly not be free from liability for actions taken after a coin flip even if that coin could be viewed in a certain light to have the power to potentially force or prevent certain actions. An AI box that tells Harvey whether to shoot or spare would be similarly irrelevant to his liability - and a scenario in which Harvey points the gun at someone and then walks away giving the AI control over the trigger is essentially no different. Harvey in all cases is responsible for constructing the scenario that (potentially) leads to someone's death and, more over, even if the gun wasn't fired because the AI decided to spare the person Harvey would be on the hook for attempted murder.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#169
post #86

Earlier quoted context omitted.

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…

I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.

But it's not totally irrelevant in this story.

Cops are already susceptible to confirmation bias, and for "efficiencies" they are delegating part of their job to apparently magical tools that will only increase their confirmation bias. And because it is for efficiency you can bet they won't be given extra time to validate the results.

What or who is at fault isn't either/or, it's a bunch of compounding factors.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#170

https://archive.is/yCaVV - Archive link to get around the paywall. https://www.theguardian.com/us-news/2026/mar/12/tennessee-gr... - Another article on this without a paywall. It's annoying that both articles are calling this AI error. This was human error, the police did the wrong thing and the people of Fargo will end up paying for this fuckup.

Human police errors are so routine that they're not news worthy.
Post reply on HN