81–90 of 153 posts
The idea that it could be used to attest the authenticity of any random person or device surely wasn't a thing serious people expected was it?
I'd always thought the usefulness of C2PA was limited to verified devices in custody by trusted actors. Like a security camera with a tamper evident enclosure, or an organisation being able to attest that they recorded the imagery. The idea that it could be used to attest the authenticity of any random person or device surely wasn't a thing serious people expected was it?
But that is also not the case, because whatever keys are in those devices may have been duplicated in the factory or somewhere along the supply chain. Or the stuff is cloud connected and an exploit can be executed via that.
Or, as written in the blog post you're commenting on, software exploits.
The whole idea is that the concept works for no one.
You could actually make this secure, by having firmware in the camera module itself do the signature before handing it off to the rest of the system. The key is to prevent ingress of control, as long as the module only emits signed photographs or video, and has tightly controlled input channels, for zoom, aperture, etc... it seems a quite reasonable project.
And, given that the main threat here is disinformation by nation state actors, they can also attack the camera module (or its supply chain) instead.
It won't stop fraud by governments and/or determined/well-resourced attackers, but it'll make it difficult enough for 99.99% of the public. And as usage drops over time, it becomes more socially acceptable to justify further limitations.
Governments and corporations (e.g. banks) will require that you use a "trusted" (meaning locked-down) devices to interact with their services. We're already seeing some companies block GrapheneOS/LineageOS.
Earlier quoted context omitted.
Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not, since if there's really all that much riding on it, people will point out it can be bypassed. Should we also abolish Pangram, because it's not 100% accurate? Someone might be convinced a text is not AI-generated when it actually is! We should get rid of it rat…
You're missing all of the points that there could be by focussing on random people. While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level. This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.
I like being contrarian as much as the next guy, but "Actually, having security is worse for security" is taking it a little too far.
Earlier quoted context omitted.
You're missing all of the points that there could be by focussing on random people. While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level. This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.
It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors. I like being contrarian as much as the next guy, but "Actuall…
It is however in the interest of the people to keep the systems running in an untainted way.
As said, on the individual level it's a tragedy, but one that can be absorbed somewhat. Democracy itself failing otoh is kinda hard to absorb.
C2PA is not "having security". It is "having an illusion of security for compliance and CYA reasons, that can be fairly trivially exploited by nation state actors". Banality of evil. Again.
___
Actually, come to think of it, "security" is the wrong term there. Signatures don't secure anything. They attest.
Those are different things. Argh and I ran with your term aah
Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…
I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing. People are concerned that the technology will lend additional credence to the last 0.1%. But anyone who thinks about the technology for 2 minutes will…
Also, note that C2PA should have something like Level 3 as well: "The image is mathematically proven to have come from the physical camera sensor."
It's somehow difficult to achieve this, but it's possible (although the attacks will always be possible of course).
Earlier quoted context omitted.
I don’t think there is a technical solution to this problem but I think there is a legal one. Make it a legal requirement to mark AI generated photos and enforce penalties for posting unmarked AI generations. Social media should also mark the country of origin for each post, with the knowledge that posts from your own country are covered by these laws.
The EU did this, Claude and Microsoft complied and HN was practically rioting about it literally yesterday : https://news.ycombinator.com/item?id=49421158
The point is making it easier to go after bot accounts that spam generated videos to influence politics. They need to disclose that its AI and lose their power or lie and get criminally prosecuted.
It's not the same thing as model providers adding a mandatory watermark to everything, and even worse one you can't verify yourself and have to trust that Anthropic is telling everyone the truth. People should have the option to use undetectable AI tools in private, even if it's illegal to post the outputs on social media.
Most people don't read HN, or understand tech, and so if the device says "captured with camera" then they will believe it since Google says it's true.
Accusing someone of a crime, with fake but verified photographic or video evidence will be trivial, and claiming it's fake just makes someone tap the "Google says it's true" sign.