This article makes me wonder: does forensic software undergo any kind of 3rd party testing for accuracy? Or are we literally just taking someone's word that it is producing accurate results?
An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
21–30 of 37 posts
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#22I wonder if anyone has ever been wrongfully convicted because of poor forensic software. Those programmers are incredibly unethical people. I'm familiar with the food-on-the-table argument, but wow, possibly getting innocent people locked up just because you don't know what you're doing is something. It can't feel good to receive the double whammy of knowing you're incompetent and that you're incompetent enough to ru…
People have literally died due to buggy software: https://en.wikipedia.org/wiki/Therac-25
I think it's reasonable to assume that people have also been wrongfully convicted due to buggy forensics software as well, given just how hard it is to get it right.
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#23What this really highlights is that lay people trust computer forensics people to be experts, and often that trust is misplaced. Unfortunately, this isn't the only example of misplaced trust. In several places, Zdziarski laments that computer forensics doesn't live up to the standards of proper "science." Yet, his trust in the rest of forensic "science" is also misplaced. Pretty much everything besides DNA forensics, maybe fingerprints, is junk: http://lst.law.asu.edu/FS09/pdfs/Koehler4_3.pdf (page 4).
Indeed, the National Science Foundation has been looking into the field and has been shocked by how unfounded it is: http://www.nsf.gov/pubs/2013/nsf13120/nsf13120.jsp ("While the report acknowledges that 'the forensic science disciplines have produced valuable evidence that has contributed to the successful prosecution and conviction of criminals as well as to the exoneration of innocent people,' it cites a need for systematic research to validate the various disciplines’ underlying assumptions and methodologies, adding that the 'forensic science ... communities will be improved by opportunities to collaborate with the broader science and engineering communities.')
All that is just a polite way of saying: "holy shit your techniques lack real scientific foundations; you guys need to hire real scientists."
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#24John's blog post sure has a hell of a lot of conjecture and opinions in it for someone so hellbent on discrediting closed source forensic tools for doing the same. John would have a lot more credibility if he would shut his mouth and stuck to reporting unbiased technical information like he asks for from everyone else.
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#25The author states they were shocked at how Sinclair seemed to get off easy, with just a slap on the wrist. I'm not sure why that's shocking. It's been painfully obvious for more than a decade that there exists a certain class of people in the US who are above the law, e.g. James Clapper.
I read that line to mean they were shocked that he received as much punishment as he did. The phrase just before that talks about damaging the prosecution's case.
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#26A sad read on many levels. "I investigated a little. As it turns out, this particular manufacturer first copies the iOS file system onto the Windows partition that their software is running on, and then pulls the timestamp information off of the copy of the data." "Irresponsible, to say the least. But this is the quality of the forensics software assisting our government and military. Poorly written, over-priced assu…
I think you should understand that America's economy has a long an illustrious history of "snakeoil barons becoming rich on bullshit". It's kind of the very basis of our whole system.
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#27What this really highlights is that lay people trust computer forensics people to be experts, and often that trust is misplaced. Unfortunately, this isn't the only example of misplaced trust. In several places, Zdziarski laments that computer forensics doesn't live up to the standards of proper "science." Yet, his trust in the rest of forensic "science" is also misplaced. Pretty much everything besides DNA forensics,…
On top of the junky aspects of forensic "science" are many bad practices. 1600+ cases in Massachusetts were called into question due to a crime lab employee who told police and prosecutors what they wanted to hear. She was a hero until she wasn't.
She was convicted of a crime and got a short prison term. The investigation found she "acted alone."
Is she an outlier?
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#28Ironically, the amount of pressure from the top involved in this (and related) cases caused several prosecutions of sexual misconduct in the military to be dismissed . You read that right. There's a doctrine called "unlawful command influence" where, if the posture of the Commander on Chief on down suggests that there is a "right verdict" independent of the facts, then the military courts should dismiss cases rather…
We have something similar going on in the UK with Cameron making commentary and publicly taking sides on court proceedings. http://www.bbc.co.uk/news/uk-england-london-25350419 http://www.bbc.co.uk/news/uk-politics-28014035
While that's not good, it's very different thing. Those two examples were of normal, civilian courts, proceeding of an independent judiciary; a President or Prime Minister has the "power of the bully pulpit", but no formal power. Our minimalist Constitution goes so far as to say Federal judge compensation "shall not be diminished during their Continuance in Office", i.e. the other branches of government can't dock their pay.
In military courts, the judges and jury are in the chain of command, so as patio11 put it, "the military courts should dismiss cases rather than having military officers be, essentially, ordered to convict." Hence the formal name "unlawful command influence".
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#29Ironically, the amount of pressure from the top involved in this (and related) cases caused several prosecutions of sexual misconduct in the military to be dismissed . You read that right. There's a doctrine called "unlawful command influence" where, if the posture of the Commander on Chief on down suggests that there is a "right verdict" independent of the facts, then the military courts should dismiss cases rather…
To be fair, the Uniform Code of Military Justice is wholly separate from the Constitution and the modern form (including the unlawful command influence clause[1]) has only really existed since 1950.
He probably should have been more wary after commenting on the Chelsea Manning case, but it's worth noting that there have been many judges who haven't dismissed cases after his remarks, so at the very least the propriety of his statement (which was fairly tame and limited to calling for discharge once convicted) is debatable.
[1] http://www.au.af.mil/au/awc/awcgate/ucmj.htm#837.%20ART.%203...
Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair
#30I wonder if anyone has ever been wrongfully convicted because of poor forensic software. Those programmers are incredibly unethical people. I'm familiar with the food-on-the-table argument, but wow, possibly getting innocent people locked up just because you don't know what you're doing is something. It can't feel good to receive the double whammy of knowing you're incompetent and that you're incompetent enough to ru…
>work on critical health software while ignorant This struck a nerve because that's how I feel in my position. I really had no idea what went into the products that I'm not working on. I feel ignorant, but someone's giving my code the okay and occasionally throwing some back, so I must be doing something right. Someone said, "He looks like he can handle this job" and here I am with no prior experience in this particu…
Not to diminish the difficulty of accomplishing this, but there is a good answer to your question: it requires both a strong culture of quality assurance (this is what you're talking about with mentors and watching each others' backs), and a strong specialization of quality assurance, made up of the experts. Unfortunately, many software organizations treat the specialization like a joke of a useless roadblock and only pay lip service to the culture. I think this may be because lots of software truly doesn't need to be very high quality, while other software truly does, and determining which kind you're making and justifying the additional cost of quality (which is enormous) is not straightforward.
In the case of this specific article, I find it odd that the blame is laid at the non-forensic-expert software engineers, rather than the companies that employ them, seemingly without the support of a tightly integrated set of experts. There is nothing wrong with dividing labor between creating software and defining and verifying what the software does. Both jobs are difficult in specialized ways. It seems that, in the opinion of one expert at least, these companies are just doing a poor job of QA.