Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Innocent woman jailed after being misidentified using AI facial recognition

grandforksherald.com

Post reply on HN
Print view View original

151–160 of 417 posts

Re: Innocent woman jailed after being misidentified using AI facial recognition

#151

Post by dml2135 »

pj_mukh wrote: post #119
rglover wrote: post #86

Earlier quoted context omitted.

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it. Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct." Don't underestimate how much authority and thinking people will delegate to machines. N…

I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.

The “I” in “AI” stands for “intelligence”. Cops are using AI facial recognition because it is being sold to them as being smarter and better than what they are currently capable of. Why are we then surprised that they aren’t second-guessing the technology?

Re: Innocent woman jailed after being misidentified using AI facial recognition

#152

Post by pj_mukh »

caconym_ wrote: post #137
pj_mukh wrote: post #133

Earlier quoted context omitted.

all accountability need not be punitive, we can certainly talk about systemic guardrails. What I find disbelief in, is someone saying the Chief of Police saying "We are not going to talk about that today?" is not the biggest scandal, but the AI is.

> someone saying the Chief of Police saying "We are not going to talk about that today?" is not the biggest scandal, but the AI is. Who is this "someone"? OP's article and the discussion here are absolutely not neglecting the human factors and general institutional failure that made this possible. But it's also true that without these "AI" tools, it would never have happened.

Yea but this feels like when a Waymo ran over a cat, and a Human driver ran over a toddler and both got the same level of coverage in the media (actually the cat got more follow-up coverage). And I'm supposed to believe both issues are equally important.

No. That's gaslighting, and totally misplaced political activation.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#153

Post by toraway »

pj_mukh wrote: post #133
caconym_ wrote: post #129

Earlier quoted context omitted.

As soon as we start to see a pattern of shitty vibe-coded software actually harming people via defects etc. (see: therac-25), I would hope that the conversation is about structural change to mitigate risk in aggregate rather than just punitive consequences for the individual programmers who are "responsible". The latter would be a fantastically stupid response and would do little or nothing to reduce future harm.

all accountability need not be punitive, we can certainly talk about systemic guardrails. What I find disbelief in, is someone saying the Chief of Police saying "We are not going to talk about that today?" is not the biggest scandal, but the AI is.

  "Among his accomplishments has been establishing the department’s Real Time Crime Center that leverages technology and data to support officers in responding more effectively to incidents," the city's release said. "Zibolski also prioritized officer wellness initiatives to strengthen mental health resources and resilience within the department. He reinstituted the Traffic Safety Team to focus on roadway safety and proactive enforcement, and ... played an active role in statewide discussions on various issues affecting law enforcement."
From the same article... He spearheaded a push to "leverage technology and data to support officers in responding more effectively to incidents", then that same technology mistakingly ruins a woman's life by passing along a hit to an officer who compared with her FB photos and said "sure, seems right".

The technology seems highly relevant here. Plus, as we've seen in the software world, when a mandate comes from the top to use the shiny new magic AI tools as much as possible, the officer may have felt pressured to make arrests using the new system they paid a bunch of money for instead of second guessing whatever it spits out.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#155

Post by fluidcruft »

JumpCrisscross wrote: post #60
rpcope1 wrote: post #10

There's no way this isn't a slam dunk case to sue the piss out of the Fargo Police, probably the US Marshals and maybe other orgs. The woman in the surveillance phone clearly looks way younger, among the many other obvious signs this woman didn't do it. I hope she wrings at least several million dollars out of the government.

“Unable to pay her bills from jail, she lost her home, her car and even her dog.” Who stole her dog?!

Probably picked up by animal control as abandoned and euthanized.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#156

Post by suzzer99 »

jmyeet wrote: post #91

We are rapidly becoming a world where every person is one inscrutable LLM decision from having their life ruined with no recourse. This type of incident isn't new and is only going to get worse. The problem is our governments are doing absolutely nothing about it. I'll give two examples: 1. Hertz implemented a system where they falsely reported cars as being stolen. People were arrested and went to jail for rental ca…

I'm banned from Amazon KDP publishing for life because a fraud detection bot hallucinated that my e-book was plagiarizing my paperback (it didn't realize they're the same book). A bunch of email appeals that I'm pretty sure were also bots went nowhere. With each appeal, the reasons for my ban got progressively more vague, until they didn't mention the plagiarism part at all, just something nonsensical about creating a negative customer experience. Evil company.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#157

Post by renewiltord »

ifh-hn wrote: post #148

Just reading the headline I said to myself: bet this is in America. Every time I see something like this I can never quite believe this sort of stuff happens. Complete, life ruining incompetence, with no consequences for the idiots that caused this to happen. Ignoring the AI input, which to me has nothing to do with this (it was used as a tool to identify a potential suspect), this woman went to jail for 5 months on…

Indeed. Something like the Post Office Scandal would never happen anywhere but in the US.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#158

Post by throwaway314155 »

caconym_ wrote: post #118
throwaway314155 wrote: post #111

Earlier quoted context omitted.

There’s nothing wrong with your comment per se, but it’s almost as if you didn’t even read the comment you’re responding to.

Let me help you out with this comprehension issue. The point of my comment is that I disagree with the apparent premise of the comment I replied to, which is that "AI" is some generic investigative tool that we can neatly snip out of the picture to blame this incident on human factors at the individual level ("the professional human-in-the-loop who shirked all responsibility"). Said comment also implies that people a…

Like I said, there wasn’t anything wrong with your comment. It just didn’t seem to directly address the parent comment. This does, thanks.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#159

Post by lotsofpulp »

dml2135 wrote: post #151
pj_mukh wrote: post #119

Earlier quoted context omitted.

I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.

The “I” in “AI” stands for “intelligence”. Cops are using AI facial recognition because it is being sold to them as being smarter and better than what they are currently capable of. Why are we then surprised that they aren’t second-guessing the technology?

Because they are supposed to possess minimum levels of intelligence found in homo sapiens, which includes not believing anything a salesperson says.

Also, their whole job is dealing with people who constantly lie to them.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#160

Post by deepsun »

wat10000 wrote: post #122
pj_mukh wrote: post #119

Earlier quoted context omitted.

I'm sorry but this is a piss-poor excuse. When I Claude code broken features, I'm responsible 100%. Why are cops not treated the same way? OP is right, AI is totally irrelevant in this story. If the point is "cops can't be trusted". Why do they have GUNS?! AI is the least of your problems. I feel like I'm going crazy with this narrative.

When are cops ever treated the same way as the rest of us?

Well in most cases I would prefer to have a cop's word to outweigh a word of an average joe.
Post reply on HN