Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

cnn.com

Post reply on HN
Print view View original

131–140 of 221 posts

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#131

Post by HDBaseT »

latexr wrote: post #84
firefoxd wrote: post #30

Without even looking at the AI part, I have a single question: Did anybody investigate? That's it. Whether it's AI that flagged her, or a witness who saw her, or her IP address appeared on the logs. Did anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. But that's not what happened, they saw the data and said "we got her". But this is the worst part of the story: > And after her o…

Yes, of course someone should have investigated, but the larger point here is that people don’t because they are being sold a false narrative that AI is infallible and can do anything. We could sit here all day arguing “you should always validate the results”, but even on HN there are people loudly advocating that you don’t need to.

I don't think people on HN think "AI is infallible", I think people on HN believe HN is sufficient enough for "most tasks". In the context of HN "most tasks" refers to programming tasks, not arresting and jailing people tasks.

You should always validate the results, but there is an inherint difference between an AI generated tool for personal use and a tool which could be used to destroy someones life.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#132

Post by zephen »

jqpabc123 wrote: post #102
zephen wrote: post #72

Earlier quoted context omitted.

Look, I'm generally considered AI's most vociferous detractor. But... > there is no way to tell if you are using it "correctly". This simply isn't true, at least in cases like this. I know common sense isn't really all that common, but why would you give more credence to an untested tool than an untested crack-addled human informant? The entire point of the informant, or the AI in this instance, is to generate leads.…

There is no "correct" way to use AI in order to avoid bad results. The only prudent approach is to assume all results are bad until proven otherwise. But this approach negates much of the incentive to pay for questionable results.

> The only prudent approach is to assume all results are bad until proven otherwise.

As is true with results from people.

> But this approach negates much of the incentive to pay for questionable results.

I'm not sure that follows. Even the crack-addled human informant has always been paid for questionable results.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#133

Post by shevy-java »

So cops used AI to attempt to investigate a crime. But, there was no crime - the arrest was wrong. Why can cops excuse themselves here for delegating their responsibilities (protecting society, allegedly that is) onto software? AI may also be written by some corporations to "tweak" this or that, see this foreign-looking guy being more likely to be AI-investigated. This is like the movie Minority Report - but stupid. IMO the courts should conclude that cops should not be allowed to use AI without having a prior, independently verified objective reasoning for any investigation. This mass sniffing that is currently going on is very clearly illegal. The current orange guy does not care about the law; see flock cameras aka spy cameras employed by the government on all car drivers at all times.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#134

Post by jmye »

themafia wrote: post #108
firefoxd wrote: post #30

Without even looking at the AI part, I have a single question: Did anybody investigate? That's it. Whether it's AI that flagged her, or a witness who saw her, or her IP address appeared on the logs. Did anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. But that's not what happened, they saw the data and said "we got her". But this is the worst part of the story: > And after her o…

> Whether it's AI that flagged her It absolutely was. There's no question of this. Now we need to ask how was the system marketed, what did the police pay for it, how were they trained to use it? > anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. Legally that amounts "hearsay" and cannot have any value. Those statements probably won't even be admissible in court without other su…

> Legally that amounts "hearsay" and cannot have any value. Those statements probably won't even be admissible in court without other supporting facts entered in first.

I just want to understand your argument: you believe that any alibi provided is hearsay, and has no legal value, and that they can't even take the statement in order to validate it? That's your position?

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#135

Post by 3eb7988a1663 »

segmondy wrote: post #55
3eb7988a1663 wrote: post #48

Earlier quoted context omitted.

It was worse than that, the reporting from an earlier story[0] ...Unable to pay her bills from jail, she lost her home, her car and even her dog. There is not a jury in the country that will side against the woman. I am not even sure who will make the best pop culture mashup - John Wick or a country song writer? (Also, what happened to journalism - no Oxford comma?) [0] https://news.ycombinator.com/item?id=47356968

You have more faith in the country than I do.

Normally, I would be a bit more grim, but people love their animals. I pray even the staunchest authoritarian would see the injustice of losing a dog.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#136

Post by suzzer99 »

tlogan wrote: post #49
suzzer99 wrote: post #38

Earlier quoted context omitted.

In the US there are no consequences for people in power failing to follow procedures, laws or regulations - except for being told to stop doing whatever illegal thing they're doing, and possibly getting sued way down the line, which gets paid by taxpayers.

From reading more into the case, it seems the issue may be related to how her lawyer handled the case. They probably did “identity challenge” arguing that she is not the right person. But from Tennessee’s perspective, she was considered the correct person to be arrested, so there was no “mistaken identity” in their system. In other words, North Dakota Wanted person x and here is person x. Once a judge in North Dakota…

Anyone involved in this who didn't immediately raise a giant stink to get this woman out of jail is partially at fault imo.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#137

Post by cortesoft »

shevy-java wrote: post #133

So cops used AI to attempt to investigate a crime. But, there was no crime - the arrest was wrong. Why can cops excuse themselves here for delegating their responsibilities (protecting society, allegedly that is) onto software? AI may also be written by some corporations to "tweak" this or that, see this foreign-looking guy being more likely to be AI-investigated. This is like the movie Minority Report - but stupid.…

What? There was absolutely a crime?

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#138

Post by cortesoft »

mememememememo wrote: post #93

Wow thought the bar for probable cause for an arrest warrant would be much higher. Especially to drag soneone from another state.

It’s a classic example of the base rate fallacy. The judge sees that a system with a seemingly high accuracy rate (like 99.999% accurate) has flagged a person, and they assume that means the person is highly likely to be guilty.

However, the system uses a dragnet approach, and is checking against millions of people. If you are checking 300 million people, that 99.999% accuracy check is going to find 3,000 people, and AT LEAST 99.96% of those people are going to be innocent.

This is why we can’t have wide, automated surveillance.

Re: Police used AI facial recognition to wrongly arrest TN woman for crimes in ND

#140

Post by belorn »

firefoxd wrote: post #30

Without even looking at the AI part, I have a single question: Did anybody investigate? That's it. Whether it's AI that flagged her, or a witness who saw her, or her IP address appeared on the logs. Did anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. But that's not what happened, they saw the data and said "we got her". But this is the worst part of the story: > And after her o…

Society went through the necessary lessons with DNA and fingerprints. Putting people in jail because the computer produce a match is a terrible idea, especially when its done by an proprietary dark box that no one really understand why it claims there is a match. It can be used as a tool of investigations to give the investigators an hint to find real more substantial clues, but using it like in fiction where the computer can act as the single truth is terrible for society and justice.

A month ago or so people on HN discussed facial recognition when looking victims and perpetrators in child exploitation material, and people were complaining that meta did not allow this fast enough. Neither the article or the people in that discussion draw any connection that the issues in this article could happen. People seemingly want to think that the lesson is "Never go back to North Dakota", as that is a much easier lesson than considering false positives in detection algorithms and their impact on a legal system that is constrained in budget, time, training and incentives.

Post reply on HN