A man spent a year in jail on murder charge that hinged on disputed AI evidence • Hacker News Board

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#161

Post by dragonwriter » Sun, Aug 22, 2021, 7:12 PM UTC

arp242 wrote: post #141

dragonwriter wrote: post #127

Earlier quoted context omitted.

> That's not really how it works Yes, it is. > false positives exist Matches (positive or negative) are not the same thing as fingerprints. Fingerprints are directly recorded facts. Matches are matters of interpretation (the fact of a match reported by a particular system is a distinct direct fact from the fingerprint itself, but again the significance is...what expert testimony exists to establish or challenge.)

The reality is that when "fingerprint evidence" is brought to court that it's not foolproof and has a sizeable margin of error that should be accounted for when considering the verdict, something that is currently often ignored. You can argue semantics all day, but that's the way things work right now. This seems like an exceedingly pedantic point to make.

> The reality is that when "fingerprint evidence" is brought to court that it's not foolproof

No evidence is foolproof. My entire point is that the issue isn't with “fingerprints” or the knowledge that juries and judges bring extrinsically (which was the explicit claim made upthread), but with the expert testimony that contextualizes fingerprints.

(This was even more clearly the problem with fiber evidence when the FBI crime lab was presenting pure bunk expert testimony in virtually every case.)

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#162

Post by DaftDank » Sun, Aug 22, 2021, 7:33 PM UTC

jdjd6 wrote: post #153

aeturnum wrote: post #148

Earlier quoted context omitted.

That's a good critique! Looking at historical data it seems like "commonly above $300,000" would be more accurate. It's worth saying that listed police salaries are far below these numbers and the officers get there through overtime and other extra payments. Thus the use of 'common' instead of average because you have to look back at what people were actually paid. Edit: I can no longer edit my original post so this…

How are supervisors and elected officials making 300k (after all benefits considered) in an area with a high cost of living equal to "police commonly make above 300k"? Second, it would be absurd to account for extra income to say they make what you're saying. That's like saying a cashier at walmart works three jobs to make an average of 70k a year, so cashiers at walmart commonly make 70k a year.

I can't remember which department, but I remember the local SF/Bay news many years ago talking about some supervisor at that department somehow making $1.5 million the year before (or something insane like that.) They explained how it was all possible, which to me seemed like he had just been with them for so long, he knew exactly the optimal way to play the system and exploit all the loopholes in it.

As for your second part, I guess it would seem unlikely for some positions, but I know a doctor who made like 3x his base salary each year through all sorts of absolutely legal, ethical, and hospital-sanctioned means. Taking every single weekend he could on call (even offering to "relieve" other doctors who had been assigned that weekend) for example allowed him to make much more than you would ever think possible.

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#163

Post by hellbannedguy » Sun, Aug 22, 2021, 7:39 PM UTC

hervature wrote: post #140

tooltower wrote: post #114

Earlier quoted context omitted.

While my stance on forensics is not as negative as the parent comment, I don't see how you are drawing such a sharp distinction between the "science" and the "interpretation". They are not separate things. When a scientist proposes a new test, they include measurement techniques as well as a method of interpretation. If a medical test is used in a way that is frequently prone to misinterpretation, it would be fair to…

I disagree. The canonical example here are p-values. Completely valid and accepted interpretation which will forever be abused. In the medical realm, we see the same things with false-positive and false-negative rates causing massive confusion. An even more recent example is COVID vaccine efficacy. I would take the bet that 90% of the general population could not give the accurate definition of what the definition of…

Just curious about the downvotes.

1. Was it he said statistics lie sometimes?

2. Or, questioned 90% efficacy of some Covid vaccines?

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#164

Post by duskwuff » Sun, Aug 22, 2021, 7:46 PM UTC

sdenton4 wrote: post #103

theelous3 wrote: post #92

Earlier quoted context omitted.

> Most forensic science is junk. I'm ignorant on the topic. Got links?

Forensics is full of gee-whiz-cool csi stuff which eventually is shown to be worthless. Polygraphs are a great example; they're still very wisely used in the US, despite piles of evidence showing they're prone to both false positives and false negatives. Sibling comments saying they aren't forensics because they've been discredited are missing the point: these are typical crime tech, and have been lately discredited,…

To add to that list: fire investigation. There have historically been a lot of myths in the field regarding forms of "proof" that a fire was caused by arson, and many of them were historically used to obtain (likely false) convictions.

https://www.abajournal.com/magazine/article/long_held_belief...

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#165

Post by Gibbon1 » Sun, Aug 22, 2021, 7:46 PM UTC

stefan_ wrote: post #6

PragmaticPulp wrote: post #3

The article says that employees of the AI company (ShotSpotter) manually reviewed and classified the sounds as gunshots: > records showed that ShotSpotter actually initially picked up what sounded like a firework a mile away, and this was later reclassified by ShotSpotter staff to be a gunshot at the intersection where and when Williams was seen on camera. So the AI didn’t even make the call. The staff did, manually.…

That's the point? These systems invariably turn out to not be any sort of AI at all, but sales people ready to make a call any way their customer (police) wants it made. Later it's presented as AI to a jury. It's basically the same scam as lots of forensics, junk science to get a desired result.

What freaks me out about forensics is no one at all is doing any validation testing. And they aren't doing blind testing either.

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#166

Post by emodendroket » Sun, Aug 22, 2021, 7:51 PM UTC

colejohnson66 wrote: post #123

emodendroket wrote: post #116

Earlier quoted context omitted.

It may very well be the case that a version of fire analysis that is scientifically rigorous can be done, but this would have nothing to do with how it is typically practiced.

Which is what @giantg2 said: > The science is generally sound (procedures and theory). Usually it's the prosecutors that present the forensics in a way that misrepresents what the results actually mean. https://news.ycombinator.com/item?id=28265884

That’s not the same claim. The practitioners who come up with the results are not operating in a scientifically sound or rigorous way. One documentary I watched about a wrongful murder conviction (a man was found to have burned down his house to kill his entire family) had a discussion of how the fire investigators would talk about “feeling the beast” and other nonsense. It is very flattering to describe this as in any way “evidence-based.”

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#167

Post by emodendroket » Sun, Aug 22, 2021, 7:52 PM UTC

CrazyStat wrote: post #151

emodendroket wrote: post #62

Earlier quoted context omitted.

I think it’s related. If it is good enough to put someone in jail there shouldn’t be any kind of need for manual overrides at all.

"Good" in what sense? Police and prosecutors don't necessarily care about their evidence being good in the sense of accurate. It just needs to convince a jury. There is a very, very, very long history of police and prosecutors using bad (in the sense of accuracy) evidence to put people in jail.

I’m making a normative claim. Obviously it does not describe reality.

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#168

Post by emodendroket » Sun, Aug 22, 2021, 7:53 PM UTC

MeinBlutIstBlau wrote: post #121

emodendroket wrote: post #54

I’m not a Luddite by any means but it is time to be more honest about the limitations of AI.

The only time I hear about people who talk about AI are average joes, ignorant journalists, generic business analysts, and upper management that don't know anything about programming. I wish I could point to these people and say "look at video game AI, we can't even make those without it cheating or following a routine to simulate human playstyles. What makes you think we can do that in real life without doing the sa…

Video games are a bad example because it’s pretty feasible to make an AI that would win every time.

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#169

Post by c_weighted » Sun, Aug 22, 2021, 7:54 PM UTC

kevin_thibedeau wrote: post #112

I get that their system may sometimes need manual classification of signals but how is it possible to change the triangulated location? That is the core part of the technology and doesn't require ML to implement. Seems like it's all just Theranos style vapor.

Locating on an echo at one more receivers rather than a perhaps quieter more direct path. Putting the wrong impulses together when computing a location. Computing the wrong start time for one or more impulses.

If you want to read up on all the ways this kind of thing can go wrong, the technical term here is "source localization."

Re: A man spent a year in jail on murder charge that hinged on disputed AI evidence

#170

Post by duskwuff » Sun, Aug 22, 2021, 7:58 PM UTC

jdjd6 wrote: post #153

aeturnum wrote: post #148

Earlier quoted context omitted.

That's a good critique! Looking at historical data it seems like "commonly above $300,000" would be more accurate. It's worth saying that listed police salaries are far below these numbers and the officers get there through overtime and other extra payments. Thus the use of 'common' instead of average because you have to look back at what people were actually paid. Edit: I can no longer edit my original post so this…

How are supervisors and elected officials making 300k (after all benefits considered) in an area with a high cost of living equal to "police commonly make above 300k"? Second, it would be absurd to account for extra income to say they make what you're saying. That's like saying a cashier at walmart works three jobs to make an average of 70k a year, so cashiers at walmart commonly make 70k a year.

> Second, it would be absurd to account for extra income to say they make what you're saying.

Except it's extra income that they make by being a police officer. A business that hires an off-duty police officer for security isn't hiring them because they're a big burly-looking dude; they're being hired because they are a police officer. An off-duty officer will typically wear their police uniform while working "off duty", and can take actions which an ordinary civilian could not (like conducting arrests).