Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C2PA Cameras Do Not Survive Contact with Reality

da.vidbuchanan.co.uk

Post reply on HN
Print view View original

121–130 of 153 posts

Re: C2PA Cameras Do Not Survive Contact with Reality

#121

Post by thaumasiotes »

dTal wrote: post #106
fightfake-ai wrote: post #88

Earlier quoted context omitted.

I agree with "I'm fairly certain this will defeat 99.9% of malicious users". Also, note that C2PA should have something like Level 3 as well: "The image is mathematically proven to have come from the physical camera sensor." It's somehow difficult to achieve this, but it's possible (although the attacks will always be possible of course).

Difficult and also solves nothing, since you can just point the camera at a screen.

Have you ever tried pointing a camera at a screen?

The screen is doing all kinds of crazy things that are not apparent to your eyes, but that show up clearly on camera.

Re: C2PA Cameras Do Not Survive Contact with Reality

#122

Post by chrisjj »

lazide wrote: post #99
chrisjj wrote: post #97

Earlier quoted context omitted.

> it makes preparing training material for Machine Learning significantly easier How?

A huge problem with ML training is the ‘ouroboros issue’ - training ML with input from other MLs breaks things in very deep (but difficult to stop/detect when it’s happening) ways due to the way the internal math works (model collapse). Right now, the Internet training set is becoming more and more contaminated with better and better generative AI images and video. It makes the models more screwed up, and makes it ve…

OK, but any reliance on a fakable signal such as this seems guarateed to invite poisoning.

Re: C2PA Cameras Do Not Survive Contact with Reality

#123

Post by akoboldfrying »

rcxdude wrote: post #95
akoboldfrying wrote: post #71

Earlier quoted context omitted.

What makes you so certain that this valuable research showing weaknesses in today's systems will render the C2PA concept useless forever? Yes, software LPEs are a risk -- as they are in every nontrivial computer system. New ones will appear, and old ones will be closed in time, as TFA acknowledges. Re hardware attacks: The (neat!) glitch injection attack the author describes in the linked "lighter" page only raises t…

The comparison to padlocks and bolt cutters is not relevant, they are quite different shapes of problems. A certificate that is weak enough that it should not be trusted in any case where it matters is actively harmful, not just mildly less helpful. (This is true in general: if you add a trust signal that is mainly just a bit of effort to broadcast, you'll find that scammers and fraudsters will show that signal much…

I guess you're making the argument that, while padlocks sometimes protect things of low value, thus justifying the use of cheap, easily broken locks in those cases, the times where authenticity of an image is important are nearly always cases of high importance, meaning that there's no case where an assertion that's actually fairly cheap to forge (assuming a similar "lighter" glitch attack works on phones, less than $1000 for a modern smartphone that may get bricked, a soldering iron and an afternoon's work) makes sense? Perhaps so. It's certainly easy to think of images or video clips of very high importance, and outside of kids uploading AI clips to Reddit and pretending they're real, I can't think of any similarly low-stakes cases for image authentication.

What do you think of my digital signature idea?

Re: C2PA Cameras Do Not Survive Contact with Reality

#124

Post by Gormo »

fwipsy wrote: post #57
mistercow wrote: post #40

Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…

I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing. People are concerned that the technology will lend additional credence to the last 0.1%. But anyone who thinks about the technology for 2 minutes will…

> I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing.

Exactly: most people don't bother editing visible watermarks out of AI-generated media, because they have little or no incentive to bother doing so. This will "defeat" the 99.9% of users who are not actually trying to do anything malicious, but will be a minor annoyance to the 0.1% of users who are actively engaging in fraud, fabrication of evidence, etc.

The upshot is that not only us this not useful for its intended purpose, it will lure people into a false sense of security by creating expectations that AI-generated media will always be easily identifiable as such, and reduce the level of scrutiny that gets applied to the stuff that actually is malicious.

Re: C2PA Cameras Do Not Survive Contact with Reality

#125

Post by senordevnyc »

hypfer wrote: post #107
akoboldfrying wrote: post #101

Earlier quoted context omitted.

There is no content in your response. At all. I honestly don't think I've come across a post this devoid of content on this site before. All I can perceive in it is a generalised hate towards some broader thing that you feel certain I'm consciously aligned with somehow, and that you think is inherently and obviously evil. Because... I don't use my real name on here? (Is your real name "hypfer"?) Because I don't have…

It is correct that I have completely side-stepped your frame and not even attempted to engage with it. This is on purpose, because wanting people to engage with that specifically engineered frame is exactly the bad faith mechanism I've described. What is interesting is what happens when these tactics are called out in a meta comment like I did, because it's actually a common thing that people start flailing like this…

What on earth is happening here? I seriously have no idea what you're even talking about at this point. You seem to be five levels of meta deep and talking in circles about yourself?

Very bizarre.

Re: C2PA Cameras Do Not Survive Contact with Reality

#126

Post by senordevnyc »

hypfer wrote: post #117
dTal wrote: post #113

Earlier quoted context omitted.

Gonna make this real short and simple: it's totally unclear what you're on about. You've described nothing, merely complained, infuriatingly, in the vaguest possible terms.

Don't take this as dismissive or a dunk or anything like that, but maybe an LLM can help with unpacking? At least I know that they understand my writing and that they can help me understand other writing I don't. I can assure you that there is a point. Whether it is worth bothering is of course your decision. I'd wager probably not, but I guess that depends. No hard feelings either way.

Maybe you should give your comments to an LLM, ask them to untangle wtf you're on about, and then have them rewrite into something coherent for the rest of us. Maybe ask for some pointers for future comments as well.

Re: C2PA Cameras Do Not Survive Contact with Reality

#127

Post by SkyBelow »

Gigachad wrote: post #59
mistercow wrote: post #40

Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…

I don’t think there is a technical solution to this problem but I think there is a legal one. Make it a legal requirement to mark AI generated photos and enforce penalties for posting unmarked AI generations. Social media should also mark the country of origin for each post, with the knowledge that posts from your own country are covered by these laws.

That hasn't done anything to stop scamming, so why would it apply to AI? People located outside of areas with these laws won't have to follow them, and this reduces building a immune system to such actions, making people more likely to fall for it when done by those not bound by the laws.

In a real like political misinformation, this will have the effect of making people trust non-watermarked images more, which will then be used by foreign actors to pass off propaganda as legitimate.

Also, if you don't hold people responsible for spreading an image they know is fake, bad actors can take advantage of this even within the US (they purposefully spread a image they have reason to think is fake but lacking a watermark), but holding people responsible for a strict liability crime for spreading AI without knowing it is AI seems an even worse route.

I'm not sure a law even makes the issue better in a 'don't let perfect be the enemy of good' sort of way.

Re: C2PA Cameras Do Not Survive Contact with Reality

#128

Post by Gormo »

fwipsy wrote: post #72
treyd wrote: post #62

Earlier quoted context omitted.

It's actually worse if it is plausibly trustworthy for "99.9%", since that's enough that naive users will get accustomed to believing the verification badge is authentic. When a motivated malicious user (who doesn't actually need that much resources) will be able to convince people something is authentic because the verification passes when it shouldn't since naive users are primed to believe it by default.

Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not, since if there's really all that much riding on it, people will point out it can be bypassed. Should we also abolish Pangram, because it's not 100% accurate? Someone might be convinced a text is not AI-generated when it actually is! We should get rid of it rat…

> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together?

Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with.

This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.

Re: C2PA Cameras Do Not Survive Contact with Reality

#129

Post by mistercow »

fwipsy wrote: post #57
mistercow wrote: post #40

Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…

I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing. People are concerned that the technology will lend additional credence to the last 0.1%. But anyone who thinks about the technology for 2 minutes will…

> I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing.

Everyone realizing that photos don't prove anything would defeat 100% of malicious users. I don't understand what people incorrectly trusting photos is supposed to achieve at this point, in your view.

Re: C2PA Cameras Do Not Survive Contact with Reality

#130

Post by account42 »

mistercow wrote: post #40

Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…

Worse than not working, this will likely be used as yet another excuse to attack computing freedom.
Post reply on HN