This kind of software should be required by law to be open source.
Yes, just like voting machines. But how do we confirm that the software which was vetted is the software that was actually _used_?
Accused murderer wins right to check source code of DNA testing kit
41–50 of 520 posts
Re: Accused murderer wins right to check source code of DNA testing kit
#42Earlier quoted context omitted.
If the code was developed in less than 8.5 years, it seems like the answer must be yes based on their previous claims.
Well I mean if something requires 8 man years of work you can do it in a year with 8 people. Not always, but usually.
I would claim that if something requires 8 man years, that it will most definitely take more than a year to develop with 8 people.
Communication takes time, coordination takes time, there is an incremental cost to each news person added to a team. From experience, perhaps with 2-3 people who happen to gel well together you may get close to proportional scaling of output, but with 8 it’s really unlikely in the real world.
Re: Accused murderer wins right to check source code of DNA testing kit
#43> Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour. So it’s definitely riddled with bugs. And I can’t imagine that much matlab code following rigorous software engineering practices.
My exact thoughts. This sounds like a classic example of launching a prototype created by domain specialists (biostatisticians and bioinformaticians) as production software and skipping on the expensive stuff, like sound development practices
For reference, whole Tesla Autopilot is 'few hundred thousand' loc.
Re: Accused murderer wins right to check source code of DNA testing kit
#44Earlier quoted context omitted.
> a forensic pathologist was giving fake prosecution evidence The doctor handled thousands of cases in his career, and a well-funded inquiry found issues with less than 1% of them. Sure we'd like that to be 0%, and society should spend time thinking of how much they rely on complicated processes of reasoning, but that's a really good accuracy.
You say that as if these were mistakes or unfortunate culminations of circumstance. The man committed fraud and sent innocent, grieving parents to prison by proxy. I don't think it's unreasonable to expect that to be 0.00% and punish those harshly who exceed that metric.
Additionally it could be made a requirement that, like in professional sports, two or three blood drafts are taken - one sample is analyzed and one is either kept until the case is closed or analyzed at a different lab if the evidence is contested.
Re: Accused murderer wins right to check source code of DNA testing kit
#45Why don't they just redo the analysis with a few other software packages?
Who is "they"
Probably "they" refer to prosecution. And yeah, why is prosecution going with that lab/software only? One reason is probably the DNA available for more testing is not enough / one testing could been done reliable with DNA taken from crime scene. Other reason might be behind the scenes incentives, which seems is the defense strategy here
Re: Accused murderer wins right to check source code of DNA testing kit
#46It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
> a forensic pathologist was giving fake prosecution evidence The doctor handled thousands of cases in his career, and a well-funded inquiry found issues with less than 1% of them. Sure we'd like that to be 0%, and society should spend time thinking of how much they rely on complicated processes of reasoning, but that's a really good accuracy.
If the error goes against innocent people, then it must be unacceptable. If the error sides with the “would-be” criminal, that is more acceptable as we, as a society, have decided that letting a few bad folks go is preferred to imprisoning innocents.
Re: Accused murderer wins right to check source code of DNA testing kit
#47> Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour. So it’s definitely riddled with bugs. And I can’t imagine that much matlab code following rigorous software engineering practices.
I have done a lot of source code review in my time. For security assessments. Our general rule of thumb was about 10k lines per week that we can really get deep on. 10 lines an hour would only be for the most dense code and critical path stuff. They will need a reviewer that knows the domain (DNA), but it’s perfectly reasonable to review that code on a weeks/months time scale, definitely not years.
Re: Accused murderer wins right to check source code of DNA testing kit
#48It's an important development. Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. They are as likely to lie as any other witness. Perhaps even moreso. "Those arguing on behalf of the defense cited past problems with other genetic testing software such as STRmix and FST (Forensic Statistical Tool). Defense expert witnesses Mats Heim…
> a forensic pathologist was giving fake prosecution evidence The doctor handled thousands of cases in his career, and a well-funded inquiry found issues with less than 1% of them. Sure we'd like that to be 0%, and society should spend time thinking of how much they rely on complicated processes of reasoning, but that's a really good accuracy.
From the Wikipedia page
> In June 2005, the Chief Coroner of Ontario ordered a review of 44 autopsies carried out by Smith. Thirteen of these cases had resulted in criminal charges and convictions. The report was released in April 2007, indicating that there were substantial problems with 20 of the autopsies.
That sounds more like 45% to me.
Also this bit from the article about a baby that allegedly had a skull fracture:
> Later exhumation of the child and examination of the skull have shown that there was no skull fracture. It is thought Dr. Smith confused the normal gap between the baby's skull plates for an injury.
Holy shit that is bad. Even as a lay person I know about these gaps.
Re: Accused murderer wins right to check source code of DNA testing kit
#49There is so much junk science going on in forensics that it would be great to require everything to be open sourced. Same for voting machines and anything police in general is using (predictive policing is pretty scary). There is way too much stuff hidden and can be challenged only if you have very deep pockets.
Re: Accused murderer wins right to check source code of DNA testing kit
#50This kind of software should be required by law to be open source.
Yes, just like voting machines. But how do we confirm that the software which was vetted is the software that was actually _used_?
Compile it, run it with the same inputs the prosecution claims was provided, and see if you get the same result. That's sorta the point of computers, right? Same inputs on the same program, same output?