61–70 of 91 posts
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…
Earlier quoted context omitted.
The cryptography to support this has been around for ever and it's been next to impossible to make the decision makers at companies and large organizations care, much less end users. Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly…
For practical uses the cryptography is quite new. Essentially if you cryptographically sign an image coming out of a sensor then you’re only getting whatever original version of it is and if you modify it even just to resize it for web for example, then it’s going to break the cryptographic chain of trust. You need to use zero knowledge proofs to allow for necessary image manipulations while keeping a signature.
Earlier quoted context omitted.
I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…
What about adding other sensors like LIDAR to make it harder to fake what you’re filming? I think about it often too…
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…
Agreed that cryptographic provenance seems like the best way forward, however the goal is more about proving authenticity, less about detecting fraud. Small but important distinction. There are industry initiatives around this already such as CAI https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat...
My take is that proving authenticity might not be something we can do with any degree of accuracy in a general sense. So if that is infeasible, then we need _some_ kind of mitigation. Something like CAI allows us to make the an assessment about the how much trust to give an informational source, probably taking into account multiple factors (known exploits in the source device, reputation of originator and what claims are be attached in the metadata). This might allow me to accept that a given video originated from a local tv station, rather than tiktoker edit, but I still need to asses if that station used genAI or has been compromised or whatever else. But that seems a much narrower reputational problem, that also will be contextual.
Earlier quoted context omitted.
I was thinking the same thing- we've had digital signing algorithms for decades, and those seem to work fine enough. There's a healthy distrust of cert authorities, sure, but it still works .
The signature only proves that the website/Bill the photographer was involved in the chain sending the website/photograph to you, not anything about the content itself. Unless you have reason to trust Bill himself you can't trust that he actually took the photo, or that it isn't ai generated. Although knowing that Bill isn't tech savvy enough to do those things might be enough.
Narrowing down the problem to "Do we trust Bill" is at least something we can attempt to address. This also eliminates some sources where we can more easily discount things. If the source is 4chan, maybe I don't need much more information to make my repetitional assessment.
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…
What's the threat vector you're trying to mitigate here? If you're wondering whether a movie that claims to be produced by Disney really was, if it's in theaters or on Disney+, then you can trust it was actually made by Disney or at least licensed to them. As long as the Washington Post still employs its own photographers and doesn't accept imagery submission from the general public, you should be able to trust a pho…
If I see a photo on twitter claiming to come from the Washington Post, it might not be.
If I see a photo in my facebook feed of a rioter, did it come from poster, or are they just reposting something else? Did that repost come from a newsource I trust, like the WP in this case, or from some reddit post, maybe edited or synthetically generated?
> Maybe they're showing you something made by AI that isn't real, but as the owner of their own signing key, nothing would prevent them from signing an AI-generated image
That's right. This only helps narrow the source down, then you still need to decide if you trust the originator. But I think a lot of the problems we've seen with social media disinformation is the wide dispersion of content claiming trustworthiness from a reputable source, falsely.
Earlier quoted context omitted.
The revocation list is for the TV. Intel revokes a TV's key, distributes the updated revocation list on new Blu-ray discs, and when a compliant Blu-ray player is playing one of those new discs it will refuse to negotiate with a revoked TV. Now that I think of it, I wonder if compliant Blu-ray players actually save the new revocation entries and then continue refusing to negotiate with revoked TVs even for old Blu-ray…
If so, couldn't a malicious disc revoke all TVs?
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…
I agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either: 1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks Note that the guarantee in (1) wouldn't preven…
My guess is likely because it seems like this would be impossible to implement without adding DRM to the smartphone and/or locking down Open Source image editors out of the attestation process. You would need to prevent access to the software, firmware, etc... otherwise the device could be virtualized or the program recompiled to circumvent the signature.
And for obvious reasons there's going to be pushback to adding that kind of DRM to smartphones. The tech does likely exist; this sounds to me like just normal attestation? It would likely hook into something like the Play Integrity API. A lot of people already hate the Play Integrity API though.
It's not the tech that's the problem, it is as you say, that people are hesitant to do it because it would require locking down the phone's software stack in a way that is widely understood by many developers and user advocates to be anti-user and in contrast to user rights to control their own devices and load their own software and/or firmware onto their devices.
I could maybe see an argument introducing some kind of signature to a raw camera input in firmware before it ever reached the user at all -- mostly just because devs seem to have given up the fight about custom firmware on a phone in general. But if you're talking about the phone signing the image after light editing like a crop has happened, at that point you're talking about moving this signature into user-space code, and while I'm sure that problem could have been explained better to you by the devs, it's not surprising to me at all that you'd get a hostile response to that suggestion because I don't see how it would be possible to do that without locking down user-space code.
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…
The cryptography to support this has been around for ever and it's been next to impossible to make the decision makers at companies and large organizations care, much less end users. Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly…
It's not the cryptography which is the problem. It's, who do you trust with the signing keys? The list inherently has to include every camera maker, despite that industry generally not having a great security culture, as well as every camera's country of origin, and every country with a security service capable of infiltrating some other country's camera maker. Which is probably all of them.
Worse, the keys have to be in the camera. Every camera. Break one of any model and you can forge images with it. Break one of any model and publish the break and you call into question every image from every camera of that type.
Then, even if a camera hasn't given up its keys, someone can use it to take a picture of a picture.
None of this requires a cryptographic break of public key signatures.
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…