181–190 of 520 posts
Even during internal review of statistical inference code, committers argue correctness through a combination of readable code, readable tests, and statistical validation on a combination of synthetic and real data. While I agree the TrueAllele source should be provided, the "zero-defects" bar is neither sufficient nor necessary for correctness of the inferences made.
It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
But let's not jump the gun here, and assume ill-intent.... to explain something that can be attributed to negligence, or simple human error.
It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
I'm not so sure we should be allowing challenges to expert witness, with no evidence. That sounds like a rich source of abuse of the system, like a big company filing thousands of briefs to delay a case. It's important for experts to use validated, tested equipment. That's clear. And by the sound of it that is happening (stories of how code was tested and found wanting confirm that; stories of double-blind testing).…
Earlier quoted context omitted.
Of course prosecutors will favour expert witnesses that will help them get convictions. Why isn’t that obvious?
Ok, maybe I’m naive, but why is that obvious? Not arguing just trying to understand. My rationale is: prosecutors have to weigh an “easy” conviction vs the possibility that labs that always guarantee results may be biased. In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promote…
It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
i think we can all agree that black boxes are not desirable. But let's not jump the gun here, and assume ill-intent.... to explain something that can be attributed to negligence, or simple human error.
Code review is not the way to validate statistical inferences. Even during internal review of statistical inference code, committers argue correctness through a combination of readable code, readable tests, and statistical validation on a combination of synthetic and real data. While I agree the TrueAllele source should be provided, the "zero-defects" bar is neither sufficient nor necessary for correctness of the inf…
Earlier quoted context omitted.
Of course prosecutors will favour expert witnesses that will help them get convictions. Why isn’t that obvious?
Ok, maybe I’m naive, but why is that obvious? Not arguing just trying to understand. My rationale is: prosecutors have to weigh an “easy” conviction vs the possibility that labs that always guarantee results may be biased. In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promote…
I am not a lawyer but as I understand the U.S. criminal justice system (which is adversarial between defense and prosecution with the judge officiating and the jury picking the winner) prosecutors are not experts in particular technical fields which is why courts allow expert witnesses. The defense is responsible for calling their own experts who can dispute other biased witnesses. If no one successfully disputes an expert witness over time the prosecutor's trust in them is very likely to grow.
A prosecutor will always claim they did due diligence by selecting a reputable expert witness up until it's proven that the witness was not in fact reliable, but claim (likely in good faith) that they had no way of knowing the expert witness was biased.
A better solution might be to mandate that expert witnesses get independently tested but again that's something a defense council could facilitate and bring into evidence. If no defense team finds it valuable to do this kind of work then it's unlikely a prosecutor will see it as useful either. Lack of challenges implies that the evidence is more or less indisputable from a legal perspective.
Code review is not the way to validate statistical inferences. Even during internal review of statistical inference code, committers argue correctness through a combination of readable code, readable tests, and statistical validation on a combination of synthetic and real data. While I agree the TrueAllele source should be provided, the "zero-defects" bar is neither sufficient nor necessary for correctness of the inf…
How much uncertainty in the system are you willing to live with if you were falsely accused of murder?
Earlier quoted context omitted.
Of course prosecutors will favour expert witnesses that will help them get convictions. Why isn’t that obvious?
Ok, maybe I’m naive, but why is that obvious? Not arguing just trying to understand. My rationale is: prosecutors have to weigh an “easy” conviction vs the possibility that labs that always guarantee results may be biased. In the event that they get caught up with a biased lab, wouldn’t it look bad for their reputation? As in, they can’t even perform due diligence on their sources, how can you trust them in a promote…
There are plenty of profit motives involved that have little to do with securing future contracts with police departments.
At the most basic - Source code reviews are expensive. They can often throw thousands of false flags that developers have to individually track down and verify are not problems.
Then there's the reputation hit of admitting a mistake - If you've had a bug for the last 5 years that makes you less reliable, that alone is likely to impact future contracts. So given the option of revealing this, admitting the mistake, and tackling the cases it impacted... OR... simply sweeping it under the rug and fixing it internally - some companies will take option number 2 (possibly most companies).
Earlier quoted context omitted.
That seems plausible. I would argue that if prosecutors and/or cops get to pick the tool, false positives would result in more sales. Just as false negatives would result in more sales if the defendant gets to make the pick.
false positives would result in more sales. That seems highly unlikely. Prosecutors, like all attorneys, talk. It wouldn’t take too many getting burned by a false positive before word would get out (regardless of any nda). For example, once an expert gets Dauberted, you’ve got to think long and hard about being the next guy to use them.