Things are only going to get murkier when leveraging ML. All of this needs to become open source.
Accused murderer wins right to check source code of DNA testing kit
111–120 of 520 posts
Re: Accused murderer wins right to check source code of DNA testing kit
#112It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
I'm not so sure we should be allowing challenges to expert witness, with no evidence. That sounds like a rich source of abuse of the system, like a big company filing thousands of briefs to delay a case. It's important for experts to use validated, tested equipment. That's clear. And by the sound of it that is happening (stories of how code was tested and found wanting confirm that; stories of double-blind testing).…
Absolutely not! Please, if you want to know why, look into expert witness testimony as a whole, possibly can start with “burn pattern experts”.
Not to mention we have a constitutional right (6th amendment) to face and cross examine witnesses against us in the US, a black box should not void that right.
Re: Accused murderer wins right to check source code of DNA testing kit
#113While this may lead to more transparent technology use in the justice system, a more realistic outcome is that everyone accused of any crime where a technology was used in the process of determining guilt (which could get quite picayune; did the prosecutor's office use Excel? Is that something the defendant could demand to check the source code on?) will use this as a way to slow-roll the process indefinitely.
Definitely if an excel macro determined my guilt based on a dataset, I would like to be able to read that macro.
Re: Accused murderer wins right to check source code of DNA testing kit
#114> The co-founder of the company, Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour. First, the defence doesn't necessarily have to evaluate all 170,000 lines. They just need to find one buggy line which could potentially overturn the result.…
> Second, even if it did take a full 8 years, is that a good reason to deny the defendant due process?
No, but the person that wants to have it analyzed will have to either spend the time themselves, or pay the expert witness for their time; it could be a costly affair.
But I think it's warranted. An independent software review, and a double blind assertion with the exact version of the software used in the conviction to test the accuracy and reliability of the application.
Re: Accused murderer wins right to check source code of DNA testing kit
#115Earlier quoted context omitted.
> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. I imagine they're also incentivised to favour a result indicating guilt.
Why? The risk to their reputation & livelihood for a false positive should temper that.
When a DNA lab issues a false positive, its CEO doesn't go to prison, nor do they get hit with a charge-back for all the tests they've done in the past year.
Their reputation doesn't matter as much as you think. Police procurement isn't based on the reputation of the vendor. It's based on whether or not the vendor delivers the results they want.
In fact, government procurement in general, isn't based on the reputation of the vendor. This is by design.
Re: Accused murderer wins right to check source code of DNA testing kit
#116Earlier quoted context omitted.
Yes, just like voting machines. But how do we confirm that the software which was vetted is the software that was actually _used_?
Compile it, run it with the same inputs the prosecution claims was provided, and see if you get the same result. That's sorta the point of computers, right? Same inputs on the same program, same output?
> Compile it, run it with the same inputs the prosecution claims was provided, and see if you get the same result. That's sorta the point of computers, right? Same inputs on the same program, same output?
One difficulty here is that the input is a real world effect. That means the answers are rarely exactly the same. The last time I looked, breathalyzers have a ~50% margin of error. So if you blow a 0.06, your BAC is really somewhere between 0.03 and 0.09. You could mod your breathalyzer to always assume that the reading was 25% higher than the reality without being noticed, as long as your breathalyzer doesn't return a result that's already 25+% higher during testing. Even if it does, your other results will be in range.
They need to just stop using breathalyzers. They're inaccurate pieces of machinery, operated by people without any medical knowledge to speak of, who have a vested interest in the readings being high. The whole thing, from top to bottom, is just a long chain of poorly aligned incentives. Field sobriety tests are even worse. The fact that your license can be revoked for refusing the test under the above grounds is nothing short of an end-run around the 4th amendment by declaring driving a "privilege, not a right" despite the fact that it is extremely hard to live in most of the US and not be able to drive. How the fuck do I survive in rural Oklahoma/Georgia/Kentucky/Texas/etc, 40 miles from the nearest bus stop, without a car? And all without any due process. It's never impacted me, but it bothers me nonetheless.
They need to force people to go in for a blood draw to establish BAC. It is an accurate test, administered by competent medical professionals, who have no vested interest in the outcome either way. Yes it does take longer. Yes, some people who were at 0.08 will be down to 0.07 by the time the test is administered. I still find that more comforting than the fact that some people at 0.06 will read at 0.09, and some people at 0.14 are going to read at 0.07.
There's also the fact that a cop who doesn't calibrate their breathalyzer is merely going to get a slap on the wrist, and a medical professional who does the same is probably going to end up with a huge malpractice suit and possibly lose their license if it was egregious in some way.
Re: Accused murderer wins right to check source code of DNA testing kit
#117Earlier quoted context omitted.
Kind of makes me wonder: if the argument is that the code is too complex to review and understand, does that mean the company is not doing code reviews themselves?
this has actually only happened to me a couple times but it has happened - someone tells me Bryan, go look at the code X did in Y, figure out if we refactor. X would then tell me - that code is really complicated is full of algorithms! I go and look at the code realize that for what it is trying to do can be cut down from 10 pages of printed code to less than 1 and it was incredibly simple what actually needed to be…
I think there's a bias towards judging things to be "clever" if they're hard to understand
It's a cliche to have a "what idiot wrote this" outburst, then realise it's your own code, because most of us have written our fair share of "clever" code
My boss explicitly stated that he doesn't want to see any "clever" or "smart" code in our product - write code based on simple fundamentals, benchmark before deciding to optimise, and be respectful in your reviews
I like my boss a lot
Re: Accused murderer wins right to check source code of DNA testing kit
#118Earlier quoted context omitted.
Why? The risk to their reputation & livelihood for a false positive should temper that.
While there have been cases of technicians guilty of systemic fudging of data, that is a tiny exception to the vast majority of honest technicians with self-respect.
Your sentimentality is not enough for a good justice system.
Re: Accused murderer wins right to check source code of DNA testing kit
#119Earlier quoted context omitted.
> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. I imagine they're also incentivised to favour a result indicating guilt.
Why? The risk to their reputation & livelihood for a false positive should temper that.
This is a long tail regulatory force, but unfortunately, does nothing to stop bad actors from jumping in and making a quick profit for as long as they can until it goes noticed.
Society also no longer has tolerance for processes that are not fully auditable and transparent, especially when it comes to our already marred justice system.
Re: Accused murderer wins right to check source code of DNA testing kit
#120> The co-founder of the company, Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour. First, the defence doesn't necessarily have to evaluate all 170,000 lines. They just need to find one buggy line which could potentially overturn the result.…
10 lines per hour? doesn't that seem painfully slow?