Live data from Hacker News

Police officer investigated for using AI to 'create evidence' in multiple cases

news.sky.com

191–200 of 210 posts

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#191
post #85

The headline evokes ideas of creating a video of a suspect perpetrating the crime but what I think is much more likely is the police officer used AI to enhance an image in a way that they considered innocuous, e.g: a photo was blurry so they “enhanced” it. Since “enhancing” is letting AI fill in the gaps it would be using AI to “create evidence”. Regardless of what they did, tampering with evidence is completely unac…

> what I think is much more likely is the police officer used AI to enhance an image in a way that they considered innocuous, e.g: a photo was blurry so they “enhanced” it Doesn't iphones do this by default? The camera isn't actually that sharp, instead it fills in the details so it looks sharp, and sometimes it adds things that were never there. Can easily see it adding a gun in a blurry photo of someone. So almost…

And it is possible that resulting image is a better match to reality than the raw data coming out of the camera .

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#193
post #39

Earlier quoted context omitted.

I don’t know the numbers, but DNA exonerations give a bit of a natural experiment (where testable evidence was preserved).

They give a floor, and that floor is too small to be useful.

The main obstacle is the denominator. How many conversations were there (before dna testing) where testable tissue was preserved? You would want to test some random sample of them against the person convicted.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#194

Earlier quoted context omitted.

It's because they're a scam. Point the camera at a forged image with a higher resolution than the camera sensor and it will make a signed copy of the unsigned forgery. That's before getting into the practical problems with securing the keys. Every camera by every manufacturer has keys in it and the attacker only needs one key from one camera, and they get to choose the model? Creating something premised on needing to…

But what about if: …the signature included the depth measured by the autofocus system across the image? …or a tiny stereo image was included to capture depth? …or a mini video in the ten seconds before and after the photo was taken? …and the key is in a tamper proof HSM? …and the key is deleted the moment the camera detects the case being taken apart? I know that it is a losing battle to try to build such hardware wh…

The more guarantees you put in place the more people believe the system is infallible and the more valuable the exploit becomes.

If "signed" photos were treated as incontrovertible truth, then you'll just have people 3d printing hyper realistic masks or something.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#195

I wonder how many people have been unjustly imprisoned between planted evidence, made up evidence, and illegal parallel construction…

Forensics is the art of manufacturing evidence to convict people.

Ok that's a bit aggressive, but it's important to understand the limits of forensic science, the broken incentives, and systemic lack of rigor in the application of various analyses and products to inculpate/exculpate suspects after the fact.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#196

Earlier quoted context omitted.

A few years ago, one of my coworkers was arrested for a domestic violence complaint. Looking into his case, I found an extremely specific lurid description of the allegations -- and then I found the same lurid description copy/pasted to every other person recently arrested for the same crime. I'm probably getting the specific terms wrong, but I did click through to see it on a government website, because my first sus…

The complaint would be by the courts, not the police.

My mistake, and that's too bad - my priors about courts were higher than the police, and this decreases them. Thanks for the correction.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#197
post #85

The headline evokes ideas of creating a video of a suspect perpetrating the crime but what I think is much more likely is the police officer used AI to enhance an image in a way that they considered innocuous, e.g: a photo was blurry so they “enhanced” it. Since “enhancing” is letting AI fill in the gaps it would be using AI to “create evidence”. Regardless of what they did, tampering with evidence is completely unac…

> what I think is much more likely is the police officer used AI to enhance an image in a way that they considered innocuous, e.g: a photo was blurry so they “enhanced” it Doesn't iphones do this by default? The camera isn't actually that sharp, instead it fills in the details so it looks sharp, and sometimes it adds things that were never there. Can easily see it adding a gun in a blurry photo of someone. So almost…

This exact situation came up during the Kyle Rittenhouse trial. The prosecution wanted to present an image of Rittenhouse. It was materially important whether or not the picture depicted Rittenhouse with a raised gun; I looked myself and couldn’t tell because it was a dark picture from a distance (if I remember correctly, it was a still from a drone video). Since the relevant part of the image was so small, the prosecution was going to zoom the photo in on an iPad, but the defense objected on the basis that the iPad zoom algorithms could be inserting detail not present in the original image.

This seemed a plausible enough objection to me. Although a fairly techy guy, I was (and am) not familiar with the specifics of Apple image processing, but at the time I had a vague awareness that Apple had been heavily advertising its use of AI algorithms to improve the quality of images. Whether that affects zooms specifically I don’t know—but it’s not an outlandish question.

The judge did an eminently reasonable thing: he disallowed the zoomed evidence on its own, but allowed it to be re‐entered if the prosecution provided an expert witness to testify that zooming the photo didn’t meaningfully change it. For this, he was pilloried by the tech media.

Take Ars Technica, for example: they used the headline “You shall not pinch to zoom: Rittenhouse trial judge disallows basic iPad feature,” and prominently displayed the judge’s words “I know less than anyone [about technology],” as if the right thing for a judge who knows nothing about technology to do would be to determine the merits of technical evidence on his own rather than ask for an expert witness. It’s not like it would have been hard for the prosecution to find one. https://arstechnica.com/tech-policy/2021/11/rittenhouse-tria...

Anyone who’s experimented with even non‐AI‐based upscalers knows that changing the algorithm can connect or disconnect catty‐cornered objects, introduce curves, and so on. I was shocked (well, not that shocked, given the heavily partisan interest in the case) that the tech media was so confident zooming an image couldn’t possibly meaningfully change it.

In the end, the image was displayed on a big‐screen TV (which probably used some other upscaling algorithm like bilinear, not that anyone in court was technical enough to point it out). The prosecution asked whether/asserted that Rittenhouse had raised his gun in the image, and Rittenhouse said it was not raised. So the exact details of how the small few pixels in the image got upscaled turned out to be pivotal in the end after all.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#198
post #93

The headline evokes ideas of creating a video of a suspect perpetrating the crime but what I think is much more likely is the police officer used AI to enhance an image in a way that they considered innocuous, e.g: a photo was blurry so they “enhanced” it. Since “enhancing” is letting AI fill in the gaps it would be using AI to “create evidence”. Regardless of what they did, tampering with evidence is completely unac…

What would the defense do with fabricated evidence? Say that evidence is fabricated? Okay, the prosecution will say it's not fabricated, now what?

The judge will decide if the evidence is admissible or not.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#199
post #101

Earlier quoted context omitted.

https://x.com/mitchcohen/status/1476951534160257026?s=20 The replies explain that it was a leaf obscuring the face

But it used AI to stitch that onto the body, a raw camera shot wouldn't look like that.

that is not what happened, no.

Re: Police officer investigated for using AI to 'create evidence' in multiple cases

#200

Earlier quoted context omitted.

It's because they're a scam. Point the camera at a forged image with a higher resolution than the camera sensor and it will make a signed copy of the unsigned forgery. That's before getting into the practical problems with securing the keys. Every camera by every manufacturer has keys in it and the attacker only needs one key from one camera, and they get to choose the model? Creating something premised on needing to…

But what about if: …the signature included the depth measured by the autofocus system across the image? …or a tiny stereo image was included to capture depth? …or a mini video in the ten seconds before and after the photo was taken? …and the key is in a tamper proof HSM? …and the key is deleted the moment the camera detects the case being taken apart? I know that it is a losing battle to try to build such hardware wh…

> the signature included the depth measured by the autofocus system across the image?

> or a tiny stereo image was included to capture depth?

These systems work by having multiple sensors to use for depth perception, so enterprising hackers write software to create two images, one for each sensor, and put some kind of lens or mirror in front of the camera to direct a different image/screen to each sensor.

The problem is fundamentally that the device is taking unsigned analog attacker-controlled input and then signing it, and is being mass produced. So whatever you're having it do, they put something that generates the same photon pattern in front of the device and you can't fix that with cryptography.

You can probably make it so that a cheap camera needs a few hundred dollars in optical glass or similar, and expensive camera needs a few thousand dollars worth, but it's hard to see how you could make it infeasible to anyone with non-trivial resources and it's also easy to mess up even worse and make it practical even for anyone with a computer and a high resolution screen or two.

> or a mini video in the ten seconds before and after the photo was taken?

Which does what if nothing in the image is expected to be moving, or the thing you're pointing the camera at is a screen rather than a piece of paper?

Also, now to verify the signature on your 50kB image you need a 2MB video? Then by default people won't distribute images that have the ability to be verified.

> and the key is in a tamper proof HSM?

Someone figures out a timing attack on the HSM or similar and now you can extract the keys from every device of that model. Happens over and over, the chances of every device getting this right are essentially zero.

> and the key is deleted the moment the camera detects the case being taken apart?

They get multiple cameras of the same model, take one apart to see how the detection works, then having figured out how it works, take the other one apart without triggering it. Or they extract the key without ever removing the case.

Also, now your phone is going to delete its keys when you remove the case to replace the battery or a cracked screen etc., or if the detection system has a false positive? Then you need some way to transfer new keys to a thing that hasn't got any, which is an even worse attack vector than not deleting the keys to begin with.

Post reply on HN