Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Innocent woman jailed after being misidentified using AI facial recognition

grandforksherald.com

Post reply on HN
Print view View original

81–90 of 417 posts

Re: Innocent woman jailed after being misidentified using AI facial recognition

#81

Post by opo »

djfobbz wrote: post #27
Blackthorn wrote: post #20

Earlier quoted context omitted.

With all the lovely qualified immunity doctrine? That's wishful thinking.

Criminal immunity? Sure. Civil immunity? Nope! She could definitely make a nice buck.

Qualified immunity doesn't apply to criminal cases. It is used to defend against civil suits. It is unfortunately very easy to find many cases where it leads to injustice. For example:

>...Abby Tiscareno, a licensed daycare provider in Utah, was wrongfully convicted of felony child abuse when a child under her care suffered brain hemorrhaging. After calling emergency services, subsequent medical tests supported these findings. However, during her trial, requested medical records from the Utah Division of Child and Family Services (DCFS) were not provided. It wasn’t until a civil suit that Ms. Tiscareno saw pathology reports suggesting the injury could have occurred outside of her care. She was granted a new trial and acquitted. Her subsequent lawsuit for due process violations, alleging that DCFS failed to provide exculpatory evidence, was dismissed due to lack of precedent indicating DCFS’s obligation to produce such evidence.

https://innocenceproject.org/news/what-you-need-to-know-abou...

Re: Innocent woman jailed after being misidentified using AI facial recognition

#82

Post by mothballed »

blagie wrote: post #61
Jtsummers wrote: post #26

Earlier quoted context omitted.

That may protect them personally, but not the city and the department itself from being sued.

Nope. https://abovethelaw.com/2016/02/criminally-yours-indicting-a... You can be arrested, indicted, and held in jail on pretrial, and there is literally no recourse. There are many other ways jail can happen without due process. Where I live: * Civil contempt. Absolutely immunity. No due process. Record is about 16 years. Having a bad day? Judge can toss you in jail. * "Dangerous." Half a year. No due process. He-sa…

>* "Insane." Psychiatric hold. Three days. Due process on paper, not in practice. Police in my town can and do use this if they don't like you.

A friend of mine was committed longer than 3 days without council or the ability to represent themselves in the hearing. Apparently the whole process of being committed is ex parte in practice in some states.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#83

Post by jolmg »

throw_m239339 wrote: post #35
tony_cannistra wrote: post #7

Completely infuriating, but more of a commentary on the sad state of incompetent power-hungry law enforcement with tools they don't know how to use than the tools themselves. Though, the question remains: are the tools built in such a way as to deceive the user into a false sense of trust or certainty? _Some_ of the blame lies on the UX here. It must.

> they don't know how to use than the tools themselves. No, the tools work perfectly as they were design to work. The problem is that the tools are flawed. Ultimately, every single of these decisions should be approved by a human, which should be responsible for the fuck up no matter what the consequences are. > _Some_ of the blame lies on the UX here. It must. No, the blame lies with the person or the group who appr…

>> are the tools built in such a way as to deceive the user into a false sense of trust or certainty? _Some_ of the blame lies on the UX here. It must.

> No, the blame lies with the person or the group who approve the usage of these tools, without understanding their shortcomings.

The person who approved the tools might've understood, but that doesn't mean the user understands. _Some_ of the reason why the user doesn't understand the shortcomings of the tool might be because of misleading UX.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#84

Post by layer8 »

zingar wrote: post #65

“Computers don’t argue” seemed charmingly wrong about how computers work until a few short years ago. https://nob.cs.ucdavis.edu/classes/ecs153-2019-04/readings/c...

This quote from a 1979 IBM training manual remains applicable:

“A computer can never be held accountable, therefore a computer must never make a management decision.”

(https://www.ibm.com/think/insights/ai-decision-making-where-...)

Re: Innocent woman jailed after being misidentified using AI facial recognition

#85

Post by abduhl »

blagie wrote: post #61
Jtsummers wrote: post #26

Earlier quoted context omitted.

That may protect them personally, but not the city and the department itself from being sued.

Nope. https://abovethelaw.com/2016/02/criminally-yours-indicting-a... You can be arrested, indicted, and held in jail on pretrial, and there is literally no recourse. There are many other ways jail can happen without due process. Where I live: * Civil contempt. Absolutely immunity. No due process. Record is about 16 years. Having a bad day? Judge can toss you in jail. * "Dangerous." Half a year. No due process. He-sa…

This is a bit hyperbolic and the exaggerations really undermine what I think is your broader point (that there is rarely recourse when you're held for short to moderate amounts of time). It is hard for me to believe that someone was held for 16 years on civil contempt without due process or that someone was held for half a year without due process after being deemed dangerous. The reason that is hard for me to believe is that the due process is implicit in the action you describe. Civil contempt is from a judge which implies that you're already in court - that's due process. Someone being labeled "dangerous" implies that a finding was made by a neutral party - that's due process.

Just because you disagree with the outcome doesn't mean that due process wasn't given.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#86

Post by rglover »

whack wrote: post #51

> According to the court documents, the Fargo detective working the case then looked at Lipps' social media accounts and Tennessee driver's license photo. In his charging document, the detective wrote that Lipps appeared to be the suspect based on facial features, body type and hairstyle and color. > Once they were in hand, Fargo police met with him and Lipps at the Cass County jail on Dec. 19. She had already been i…

> How is this the fault of AI? It flagged a possible match. A live human detective confirmed it.

Because we're seeing the first instances of what reality looks like with AI in the hands of the average bear. Just like the excuse was "but the computer said it was correct," now we're just shifting to "but the AI said it was correct."

Don't underestimate how much authority and thinking people will delegate to machines. Not to mention the lengths they'll go to weasel out of taking responsibility for a screw up like this (saw another comment in this thread about the Chief of Police stepping down but it being framed as "retirement").

Re: Innocent woman jailed after being misidentified using AI facial recognition

#87

Post by janalsncm »

Pxtl wrote: post #41
throw_m239339 wrote: post #35

Earlier quoted context omitted.

> they don't know how to use than the tools themselves. No, the tools work perfectly as they were design to work. The problem is that the tools are flawed. Ultimately, every single of these decisions should be approved by a human, which should be responsible for the fuck up no matter what the consequences are. > _Some_ of the blame lies on the UX here. It must. No, the blame lies with the person or the group who appr…

I miss the days of earlier AI image-recognition software that would emit a confidence percentage. New LLM-related AIs are all supremely confident in every assertion, no matter how wrong.

I don’t know what tool they used, but it was very likely not an LLM. They probably have some database of drivers’ licenses and they ran a similarity search against the surveillance footage. This poor lady happened to be the top match.

Even if it also output a score, that score depends on how the model was trained. And the cops might ignore it anyways.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#88

Post by ImPostingOnHN »

tony_cannistra wrote: post #68
ImPostingOnHN wrote: post #32

Earlier quoted context omitted.

> are the tools built in such a way as to deceive the user into a false sense of trust or certainty? _Some_ of the blame lies on the UX here. It must. Are AI code assist tools built in such a way as to deceive the user into a false sense of trust or certainty? Very much so (even if that isn't a primary objective). Does any part of the blame lie on the UX if a dev submits a bad change? No, none. You are ultimately, so…

I agree with all that. Maybe the word isn't "blame," then. Surely there must be some code, perhaps moral or ethical, but ideally more rigorously enforcible, which ought to prevent the development of intentionally deceiving tools. Sure you could say this about all software, but that which can cause actual physical harm ought to be held to a higher standard.

Yes, unfortunately technology is advancing faster than the average human brain evolves more neurons, so it will only become less comprehensible to the average person.

That's setting aside the tendency for police to hire from the left side of the bell curve to avoid independent thinkers that might question authority, refuse to do bad shit, etc.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#89

Post by mothballed »

abduhl wrote: post #85
blagie wrote: post #61

Earlier quoted context omitted.

Nope. https://abovethelaw.com/2016/02/criminally-yours-indicting-a... You can be arrested, indicted, and held in jail on pretrial, and there is literally no recourse. There are many other ways jail can happen without due process. Where I live: * Civil contempt. Absolutely immunity. No due process. Record is about 16 years. Having a bad day? Judge can toss you in jail. * "Dangerous." Half a year. No due process. He-sa…

This is a bit hyperbolic and the exaggerations really undermine what I think is your broader point (that there is rarely recourse when you're held for short to moderate amounts of time). It is hard for me to believe that someone was held for 16 years on civil contempt without due process or that someone was held for half a year without due process after being deemed dangerous. The reason that is hard for me to believ…

Yeah it's "due process." In civil contempt the judge is a witness and prosecutor in the very "process" they're judging. That's the most perverted form of due process imaginable.

A judge should have to recuse themselves if they are acting as witness to the supposed infraction.

Re: Innocent woman jailed after being misidentified using AI facial recognition

#90

Post by Jtsummers »

blagie wrote: post #61
Jtsummers wrote: post #26

Earlier quoted context omitted.

That may protect them personally, but not the city and the department itself from being sued.

Nope. https://abovethelaw.com/2016/02/criminally-yours-indicting-a... You can be arrested, indicted, and held in jail on pretrial, and there is literally no recourse. There are many other ways jail can happen without due process. Where I live: * Civil contempt. Absolutely immunity. No due process. Record is about 16 years. Having a bad day? Judge can toss you in jail. * "Dangerous." Half a year. No due process. He-sa…

I don't know what you're responding to, but I don't think it's my comment.

Qualified immunity protects individuals, not departments, from liability.

The particular thread (in this thread) that I was responding to:

>> I hope she wrings at least several million dollars out of the government.

> With all the lovely qualified immunity doctrine? That's wishful thinking.

I was responding to the claim that qualified immunity protected the government, it does not.

Post reply on HN