Innocent woman jailed after being misidentified using AI facial recognition • Hacker News Board

Re: Innocent woman jailed after being misidentified using AI facial recognition

#281

Post by fc417fc802 » Fri, Mar 13, 2026, 4:58 AM UTC

caconym_ wrote: post #264

fc417fc802 wrote: post #231

Earlier quoted context omitted.

> Let me state what should be obvious: without AI (as in, the facial recognition systems involved in this case), this woman would not have sat in jail for 5 months, or indeed for any length of time at all. How do you arrive at that conclusion? Because it happened, and it wasn't an AI overseeing (the lack of) due process. The police identifying suspects is part of their job. So are arrest warrants and all the rest of…

> How do you arrive at that conclusion? Because it's beyond obvious? How would this woman have ended up in jail if she hadn't been misidentified by the facial recognition software in use by the Fargo police? She lives 3 states over; would be a hell of a coincidence if some other avenue of investigation led them to her. > I honestly don't see what AI had to do with anything here. You honestly don't see what facial rec…

It isn't obvious, no. If I drop a hammer on my foot and break my toe I can't then blame the hardware store or the manufacturer. If the store didn't carry hammers I wouldn't have been able to purchase it, I think to myself. Then I couldn't possibly have dropped it on my foot, thus my toe wouldn't be broken right now. It's a specious line of reasoning.

It doesn't matter in the slightest by what means she was selected to "win" this particular lottery. The tool rolling the dice isn't to blame. Tools (and people!) will occasionally return spurious results. Any system needs to be set up to deal with that.

So no, I honestly don't see what facial recognition software has to do with gross negligence and process failure on the part of multiple government agencies.

> without taking reasonable care to ensure said representation is correct in each and every case.

Only if that was part of the contract. Was the product delivered according to specification or not?

What if ICE used FOSS tools to put together the list themselves? Are the tools still to blame? That would obviously be absurd.

The only way the provider (never the tool) could be at fault would be something such as willful negligence or knowingly and intentionally attempting to manipulate the user's actions to some end.

What you don't seem to understand is that human negligence can't be foisted off on tools. Of course an abuser will try to play his actions off as legitimate. That isn't the fault of the tool, it's the fault of the abuser. It isn't up to an app to determine the legitimacy of LEO agent actions. Neither is it the responsibility of an arbitrary, fungible government contractor to oversee ICE.

I think you're confusing the morality of participating in a broader ecosystem with moral culpability for the process failure associated with a specific event. You can advance a reasonable argument that AI companies that choose to do business with ICE are making an at least moderately immoral decision. However that doesn't place them at fault for the specific process failures of any particular event that happens.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#282

Post by croes » Fri, Mar 13, 2026, 5:04 AM UTC

whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

Where does it say that AI is blamed.

It says she was misidentified using facial recognition.

That’s exactly what happened

Re: Innocent woman jailed after being misidentified using AI facial recognition

#283

Post by Retric » Fri, Mar 13, 2026, 5:06 AM UTC

fc417fc802 wrote: post #232

Retric wrote: post #117

Earlier quoted context omitted.

Seems like a direct response to me. >> How is this the fault of AI? > This particular "AI bogeyman" isn't just AI; it's cops with AI You can’t separate the thing from how it will be used. It’s like arguing that cars on their own aren’t particularly dangerous, but the point of buying a car is to use it thus risking the general public.

But you can in fact argue exactly that. If (arbitrary example) pedestrians are being killed due to poor road engineering practices it isn't reasonable to point at cars and say "see those are the root problem" when in fact it's due to a willful lack of sidewalks or marked crossings or whatever. Being adjacent to something bad doesn't equate to being the root cause.

History shows the timeline of dependence here. Before the introduction of cars, “poor road engineering practices” wouldn’t result in those deaths. So clearly it’s cars that are necessitating sidewalks, etc.

Same deal here, if something “becomes a problem” because of the introduction of AI, it’s AI that is the root case of the resulting issues. Many people are tempted to argue that flawed humans can’t implement the perfect system that is Anarchy, Communism, Recycling programs, or whatever but treating systems as needing to operate on the real world is productive where complaining about humans isn’t.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#284

Post by caconym_ » Fri, Mar 13, 2026, 5:11 AM UTC

fc417fc802 wrote: post #281

caconym_ wrote: post #264

Earlier quoted context omitted.

> How do you arrive at that conclusion? Because it's beyond obvious? How would this woman have ended up in jail if she hadn't been misidentified by the facial recognition software in use by the Fargo police? She lives 3 states over; would be a hell of a coincidence if some other avenue of investigation led them to her. > I honestly don't see what AI had to do with anything here. You honestly don't see what facial rec…

It isn't obvious, no. If I drop a hammer on my foot and break my toe I can't then blame the hardware store or the manufacturer. If the store didn't carry hammers I wouldn't have been able to purchase it, I think to myself. Then I couldn't possibly have dropped it on my foot, thus my toe wouldn't be broken right now. It's a specious line of reasoning. It doesn't matter in the slightest by what means she was selected t…

If you don't agree that facial recognition software is involved in a case of a woman being misidentified by facial recognition software then there is no point in me spending any more time/effort in conversation with you. Goodbye.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#285

Post by plorg » Fri, Mar 13, 2026, 5:12 AM UTC

whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

I think the biggest problem is that the popular narratives about AI enable this like of accountability sink.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#286

Post by discordance » Fri, Mar 13, 2026, 5:18 AM UTC

blitzar wrote: post #99

whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

computer said yes

lgtm

Re: Innocent woman jailed after being misidentified using AI facial recognition

#287

Post by fc417fc802 » Fri, Mar 13, 2026, 5:21 AM UTC

Retric wrote: post #283

fc417fc802 wrote: post #232

Earlier quoted context omitted.

But you can in fact argue exactly that. If (arbitrary example) pedestrians are being killed due to poor road engineering practices it isn't reasonable to point at cars and say "see those are the root problem" when in fact it's due to a willful lack of sidewalks or marked crossings or whatever. Being adjacent to something bad doesn't equate to being the root cause.

History shows the timeline of dependence here. Before the introduction of cars, “poor road engineering practices” wouldn’t result in those deaths. So clearly it’s cars that are necessitating sidewalks, etc. Same deal here, if something “becomes a problem” because of the introduction of AI, it’s AI that is the root case of the resulting issues. Many people are tempted to argue that flawed humans can’t implement the pe…

Well I (thought it was obvious that) I was referring to roads constructed relatively recently. If cars necessitate sidewalks and the city chooses to cut costs by not putting those in that isn't the fault of automobile designers or manufacturers or dealers or private owners or whoever.

To your example, technology changes and that necessitates infrastructure changing. That doesn't mean that fault for mishaps in the meantime can be attributed to the new technology. A user operating the new technology in an obviously unsafe manner is solely at fault for his own negligence.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#288

Post by abustamam » Fri, Mar 13, 2026, 5:26 AM UTC

pj_mukh wrote: post #152

caconym_ wrote: post #137

Earlier quoted context omitted.

> someone saying the Chief of Police saying "We are not going to talk about that today?" is not the biggest scandal, but the AI is. Who is this "someone"? OP's article and the discussion here are absolutely not neglecting the human factors and general institutional failure that made this possible. But it's also true that without these "AI" tools, it would never have happened.

Yea but this feels like when a Waymo ran over a cat, and a Human driver ran over a toddler and both got the same level of coverage in the media (actually the cat got more follow-up coverage). And I'm supposed to believe both issues are equally important. No. That's gaslighting, and totally misplaced political activation.

What do you propose we do in the latter situation? The news isn't the value of the life that was (presumably lost). The news is the circumstances that made that loss possible. Human driver was maybe careless, or maybe didn't look. The child safety classes I took emphasized over and over again to look around your car and yard before backing your car out. This is a problem with a known solution that unfortunately still happens despite the best efforts to prevent it.

Waymo hitting a cat is obviously less tragic, but if it can hit a cat, what else can it hit? A toddler? A human? The wall of your kitchen? This is a problem that has no known solution; furthermore, it's a problem that the engineers at Waymo don't seem overly keen on solving quickly.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#289

Post by dqv » Fri, Mar 13, 2026, 5:28 AM UTC

andrepd wrote: post #167

rglover wrote: post #86

Earlier quoted context omitted.

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…

It's not even just incompetence, but malice. "AI says so" is going to be the perfect catch-all excuse for literally everything anyone might want to do that they shouldn't. You know how techbros love to excuse every horrifying outcome of their torment nexi with "don't blame me, the algorithm did it"? It's going to be like that, but now everyone can do it.

It's also why people start parroting the phrase "the purpose of a system is what it does". Look at where we are right now: a precipice before this becomes widely used in all forms of policing. We still have a chance to police the police's use of the AI.

The purpose of using AI to identify suspects in criminal cases is to ease the burden of manual searching for a suspect (or insert whatever the purpose of statement you want). Ok, but we're getting false positives that are damaging people's lives already in the early stages. And I don't want to hear "trust me bro, it will get more accurate" as an excuse to not regulate it.

At a minimum, we should enshrine the right to appeal AI and have limits on how it can be used for probable cause.

This isn't even the only recent case of this happening. There was another case of mistaken identity due to AI. [0] Sure 4 hours isn't the same as 5 months, but still this guy wanted to show multiple forms of ID to prove who he was! The bodycam footage was posted a few months back but never got traction here.

Like if the police officer can't read numbers, they can't do breathalyzer tests on people. If the AI can't be used responsibly, then it can't be used at all.

[0]: https://www.youtube.com/watch?v=lPUBXN2Fd_E

Re: Innocent woman jailed after being misidentified using AI facial recognition

#290

Post by scarecrowbob » Fri, Mar 13, 2026, 5:28 AM UTC

foxglacier wrote: post #185

worik wrote: post #183

Earlier quoted context omitted.

So what??? This woman lost most of her material possessions, was terrorised by "goons"... The police do this stuff regularly, as black people, immigrants, "white trash" etcetera know well. Another opportunity, presented BY AI models for more routine police oppression As the wise singer said: "Fuck the police!"

Exactly, it's the police's fault, as well as the wider system they operate in that enables that kind of abuse, and they do it anyway even with out AI.

You're right that they often do a lot of harm.

The point that you're missing is that, in a system where such abuses are possible, many of us really don't want one more tool in their box for them to fuck us with.

Like, they already prove themselves incompetent- giving the power to track anyone in the US via a distributed ALPR system just makes them more dangerous. Giving them all these "AI" based tools does the same.