Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Innocent woman jailed after being misidentified using AI facial recognition

grandforksherald.com

Post reply on HN
Print view View original

261–270 of 417 posts

Re: Innocent woman jailed after being misidentified using AI facial recognition

#261

Post by throwaway17_17 »

RobRivera wrote: post #57
whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

I think it's more nuanced; it is one error in a Tragedy of Errors.

This was not a series of errors, this is (as a statistical inference) the system working as designed. This is not uncommon, it is not unplanned. The extradition of suspects from State to State is designed legislatively to function this way.

I also think there is more nuance to this situation than AI bad // Human Bad :: choose one. But while a tragedy, the ‘correct’ functioning of a system that produces tragedy doesn’t make that function and error.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#262

Post by silisili »

phendrenad2 wrote: post #249
silisili wrote: post #209

Earlier quoted context omitted.

> After a couple million dollar lawsuits the city or state will learn to be more careful with their methods You'd think, but watching how many millions my local police department and city paid out every single year leads me to believe they just don't care.

How many, exactly? Anyone can wave vagueness around. Do you have numbers or no?

I haven't lived there in years, nor do I have exact numbers, but they make national news enough for the same problem nearly every year. I'll drop you some links if you care.

1 - 38 million between 2017 and 2022.

2 - 29 million in 2023.

3 - 12 million in settlements in 2025.

Dare I keep going?

[1]https://www.wdrb.com/in-depth/louisville-payouts-for-police-...

[2]https://www.aol.com/louisville-paid-least-29m-settle-1030450...

[3]https://www.courier-journal.com/story/news/local/2026/02/04/...

Re: Innocent woman jailed after being misidentified using AI facial recognition

#263

Post by throwaway17_17 »

themafia wrote: post #66
whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

> How is this the fault of AI? It could be the fault of the company that's selling this service. They often make wildly inaccurate claims about the utility and accuracy of their systems. [0] > There's a reason why we don't let AI autonomously jail people. Yes we do. [1] > and a criminal justice system that thinks it is okay to jail people for 5 months before even starting to assess their guilt. Her guilt was assessed…

To clarify one point, her not having a bail is a function of the way interstate ‘fugitive’ warrants are designed. The Court in Tennessee had no ability to set bail, and until she entered the physical custody of North Dakota she can not have bail set.

Also, her guilt was not assessed in any common meaning of the term. The requirement for holding a person in custody, with or without bail, is probable cause. The only thing assessed was did law enforcement present a statement to a Judge that was possible to be believed in the light most favorable to the prosecution.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#264

Post by caconym_ »

fc417fc802 wrote: post #231
caconym_ wrote: post #118

Earlier quoted context omitted.

Let me help you out with this comprehension issue. The point of my comment is that I disagree with the apparent premise of the comment I replied to, which is that "AI" is some generic investigative tool that we can neatly snip out of the picture to blame this incident on human factors at the individual level ("the professional human-in-the-loop who shirked all responsibility"). Said comment also implies that people a…

> Let me state what should be obvious: without AI (as in, the facial recognition systems involved in this case), this woman would not have sat in jail for 5 months, or indeed for any length of time at all. How do you arrive at that conclusion? Because it happened, and it wasn't an AI overseeing (the lack of) due process. The police identifying suspects is part of their job. So are arrest warrants and all the rest of…

> How do you arrive at that conclusion?

Because it's beyond obvious? How would this woman have ended up in jail if she hadn't been misidentified by the facial recognition software in use by the Fargo police? She lives 3 states over; would be a hell of a coincidence if some other avenue of investigation led them to her.

> I honestly don't see what AI had to do with anything here.

You honestly don't see what facial recognition software had to do with a woman being misidentified by facial recognition software?

> Suppose ICE busts down someone's door, drags them off, holds them in an internment camp for months, and then finally goes "oh, oops, guess you were a citizen all along sorry about that" and releases them. We don't blame the source of their faulty hit list.

I actually am completely willing to blame any entity that supplies ICE with the names of people it can reasonably assume will be targeted for "enforcement action" due to said entity representing said names as being legitimate targets for said enforcement action, without taking reasonable care to ensure said representation is correct in each and every case.

What you don't seem to understand is that these abuses of law enforcement authority are predicated on at least an appearance of legitimacy, which can be provided by (e.g.) an app with (presumably) a very official looking logo that agents can point at somebody to get a 'CITIZEN' or 'NOT CITIZEN' classification. It is upon this kind of basis that they perform illegal arrests. All parties—the app vendor and ICE, as well as the people who are meant to be overseeing ICE and providing accountability—are complicit enablers in these crimes. To absolve the vendors who provide the software knowing full well what it will be used for, what its limitations are, and how unlikely it is that ICE personnel will understand those limitations and work around them to keep everything legal, is totally absurd.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#265

Post by throwaway17_17 »

throwaway173738 wrote: post #198
unethical_ban wrote: post #195

Earlier quoted context omitted.

Someone from the government should be in jail for this kind of oversight.

I think the taxpayers owe this lady at least a couple million if not more for the inconvenience they chose to put her through.

I agree, but our system doesn’t value things that way. Texas, which is one of the highest paying States for cases where intentional, fraudulent, or grossly negligent actions result in wrongful incarceration pay $80,000 dollars per year a person is locked up. But the caveat is that time only starts counting after you are sentenced, so wouldn’t even apply in TFA’s case.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#266

Post by dragonwriter »

whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

> How is this the fault of AI

It isn't, the article doesn't claim (or even imply) that it is "the fault" of AI, only that AI was part of the chain of events, and nothing is the fault of AI until AI is sufficiently advanced to constitute a moral actor. “At the source of every error which is blamed on the computer, you will find at least two human errors, one of which is the error of blaming it on the computer” remains true.

OTOH, it is potentially the fault of the reliance human actors put on an AI determination.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#267

Post by dragonwriter »

catlikesshrimp wrote: post #189
foxglacier wrote: post #179

Earlier quoted context omitted.

So what? There were false arrests and convictions made by misuse of line-ups, DNA, eye-witnesses, photos, bloodstains, fingerprints, etc. since forever. You must also blame all those other technologies, so what do you think the police should use to find suspects? In your view, the more help police have, the worse a job they'll do. Is that actually the trend?

With all other proof you mentioned, there was always a human putting his signature. Now that they can blame "AI" no specific officer(s) will take the blame, ever. If no one is responsible there will be many more false positives. And false positives destroy lives

> With all other proof you mentioned, there was always a human putting his signature.

There was a human doing that in this case; AI doesn’t inititiate charges. “In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color.”

Re: Innocent woman jailed after being misidentified using AI facial recognition

#268

Post by throwaway17_17 »

JumpCrisscross wrote: post #221
fluidcruft wrote: post #210

Earlier quoted context omitted.

It's pretty common when a dog is abandoned. Likely her children couldn't afford to care for it. I suppose there is a chance they put it up for adoption (same outcome is likely).

Sorry. I really fucking hate this. I don’t imagine there is a charity somewhere that works against this?

No large scale orgs that I know of. Our local bar has an attorney who does work against it, she has her number at the jail where other inmates will pass her number around if some mentions their dog, and intake officers will often suggest to inmates that if they have pets to call her. She is absolutely the most hard core lover of dogs I have ever met, and she will literally drive/run into danger to get to a canine to get it to a local non-kill shelter.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#269

Post by dragonwriter »

foxglacier wrote: post #185
worik wrote: post #183

Earlier quoted context omitted.

So what??? This woman lost most of her material possessions, was terrorised by "goons"... The police do this stuff regularly, as black people, immigrants, "white trash" etcetera know well. Another opportunity, presented BY AI models for more routine police oppression As the wise singer said: "Fuck the police!"

Exactly, it's the police's fault, as well as the wider system they operate in that enables that kind of abuse, and they do it anyway even with out AI.

AI is, in this case, a tool enabling it, because trawling large databases using AI allows finding people with a degree of similarity to a suspect that would reasonably constitute probable cause int he context of what was until fairly recently the norm for police work because that work relies on proximity and connections to the crime. The understanding of probable cause and what is necessary for it , given the actual investigative process in the case, including the use of large databases unconnected with the events and locality of the crime needs to adapt.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#270

Post by throwaway17_17 »

ugh123 wrote: post #161
rectang wrote: post #2

> facial recognition showed she was the main suspect in what Fargo police called an organized bank fraud case. > Her bank records showed she was more than 1,200 miles away, at home in Tennessee at the same time police claimed she was in Fargo committing fraud. > Unable to pay her bills from jail, she lost her home, her car and even her dog

The Fargo DA should be fired, at a minimum.

I can almost guarantee that the Fargo DA’s office had zero idea this happened and had never heard about this investigation before the news story. At this stage in a “case” it’s completely on law enforcement and there is no involvement by a DA’s office for the arrest warrant or the extradition order and warrant that led to this situation.

Not a fan of DA’s offices in general (they are the “evil twin” to my particular line of work after all), but realistically this one isn’t on them.

Post reply on HN