Live data from Hacker News

An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

zdziarski.com

31–37 of 37 posts

Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

#31
post #4

I wonder if anyone has ever been wrongfully convicted because of poor forensic software. Those programmers are incredibly unethical people. I'm familiar with the food-on-the-table argument, but wow, possibly getting innocent people locked up just because you don't know what you're doing is something. It can't feel good to receive the double whammy of knowing you're incompetent and that you're incompetent enough to ru…

My sources say: yes. There exists software which is designed to confuse them and convincingly plant evidence; and while that software is imperfect, they are powerless against that.

I've been a digital forensics software developer for a decade. There's plenty of buggy software, yes, but I've never heard of software that deliberately plants evidence.

FLETC and other training classes instill in investigators the necessity of manually verifying and testing their results for their reports and/or court testimony.

Most cases involve child pornography. Most of the time its known child pornography (i.e., there's a known victim, there are likely other folks who have been convicted of possessing the images). Most of the time the suspect takes a plea bargain.

It's the rare case that involves figuring out who-did-what-when events on a computer based on trace artifacts. So, the good news is that they are not the norm; the bad news is that only the best examiners (on either prosecution or defense) can typically handle them.

Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

#32
post #4

I wonder if anyone has ever been wrongfully convicted because of poor forensic software. Those programmers are incredibly unethical people. I'm familiar with the food-on-the-table argument, but wow, possibly getting innocent people locked up just because you don't know what you're doing is something. It can't feel good to receive the double whammy of knowing you're incompetent and that you're incompetent enough to ru…

All the time.

If you want to be horrified, read about the ludicrous bullshit that is a breathalyzer. The fundamental problem seems to be three fold: first, as anyone not an idiot would expect, animals process alcohol differently so at best you have a model relating alcohol intake with alcohol present in the lungs. Second, the programming appears to be ludicrously bad [1]. Pigs and manufacturers have fought and fought to avoid having to share code behind these devices or present any proof that they measure what they purport to measure besides, "Trust me." When competent engineers have finally gotten their hands on code, you'll be unsurprised to discover it's full of bugs. Third, even accepting the bac to gas model arguendo, it's an open question if these devices measure what they purport to measure.

Just a few representative issues when competent software engineers have got their hands on the code:

   - Of the available 12-bits of A/D precision, just 4-bits (most-significant)
   are used in the actual calculation. This sorts each raw blood-alcohol reading
   into one of 16 buckets. (I wonder how they biased the rounding on that.)
   - Out of range A/D readings are forced to the high or low limit. This must 
   happen with at least 32 consecutive readings before any flags are raised.
   - There is no feedback mechanism for the software to ensure that actuated 
   devices, such as an air pump and infrared sensor, are actually on or off 
   when they are supposed to be.
   - The software first averages the initial two readings. Then it averages the 
   third reading with that average. Then the fourth reading is averaged in, 
   etc. No comments or documentation explains the use of this formula, which 
   causes the final reading to have a weight of 0.5 in the final value and the 
   one before that to have a weight of 0.25, etc.
   - Out of range averages are forced to the high or low limit too.
   - Static analysis with lint produced over 19,000 warnings about the code 
   (that’s about three errors for ever five lines of source code).
I would suggest we move to an impairment model of punishment rather than a blood alcohol. Regardless, it's a mockery of a justice system to have defendants convicted based on the flawed source code of a flawed model of human biology, that prosecutors have successfully loopholed their way into not having to defend, and that has essentially no publicly available evidence proving it measures what it purports to measure.

see also http://digital.law.washington.edu/dspace-law/bitstream/handl...

[1] http://embeddedgurus.com/barr-code/2009/11/breathalyzer-sour...

edit: and just so we're clear, drunk drivers are utter assholes. But it's a basic principle that you need a high likelihood of guilt to punish people. These breathalyzers seem like utter bullshit, used mostly as a pseudo-scientific gloss to help pigs + prosecutors punish those whom they wish to punish.

Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

#33
post #13

So, I read through the article and could not find what was concretely wrong, technically, with the data. Does he state that any where? Thanks.

Yeah, he doesn't really discuss it. It's just the usual mix of crypto-anarcho/libertarian/fundamentalist paranoia I've come to expect.

He may indeed not be able to reveal certain details of the case. However, he'd certainly be capable of reproducing whatever bugs he found in the unnamed software (likely EnCase or FTK) and discussing them. Instead of science, though, he merely provides lip service thereto, along with political commentary, irrelevant asides, and run-on sentences.

Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

#34

This article makes me wonder: does forensic software undergo any kind of 3rd party testing for accuracy? Or are we literally just taking someone's word that it is producing accurate results?

There's some, but it's far too little to be comprehensive. Some law enforcement labs have entire QA departments dedicated to validating internal procedures, including how particular tools should be used.

Re: An Example of Forensic Science at Its Worst: US v. Brig. Gen. Jeffrey Sinclair

#36
post #13

So, I read through the article and could not find what was concretely wrong, technically, with the data. Does he state that any where? Thanks.

The details are of course confidential. He says: "There are reasons I’m not going to dig into the details of the case. Certain people that were involved could easily be pinpointed by revealing technical details that could be pieced together with news reports, and help build a story in your mind that would probably be inaccurate. The details aren’t so important as the errors that were made. All you need to know from a…

"The details are of course confidential. He says:"

I kept having to skip paragraphs to get to the juicy bits. Eventually, I gave up and closed the tab. I don't much like this guy's writing style, as it constantly goes off on tangents that don't really add anything to the story, much less the baity title.

Post reply on HN