91–100 of 148 posts
This was a technology that required international governmental prescription, not voluntary or viral adoption. Either way, it was horrifically flawed from the outset.
Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…
The problem is not identifying AI generated media
The problem is identifying real media. That it can actually do.
Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…
I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing. People are concerned that the technology will lend additional credence to the last 0.1%. But anyone who thinks about the technology for 2 minutes will…
Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…
What makes you so certain that this valuable research showing weaknesses in today's systems will render the C2PA concept useless forever? Yes, software LPEs are a risk -- as they are in every nontrivial computer system. New ones will appear, and old ones will be closed in time, as TFA acknowledges. Re hardware attacks: The (neat!) glitch injection attack the author describes in the linked "lighter" page only raises t…
(This is true in general: if you add a trust signal that is mainly just a bit of effort to broadcast, you'll find that scammers and fraudsters will show that signal much more reliably than honest actors. For example, every email spammer has DMARC and DKIM set up absolutely perfectly)
C2PA was never going to work to prevent abuse, but what it will do it give false confidence. Most people don't read HN, or understand tech, and so if the device says "captured with camera" then they will believe it since Google says it's true. Accusing someone of a crime, with fake but verified photographic or video evidence will be trivial, and claiming it's fake just makes someone tap the "Google says it's true" si…
Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful. You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to cas…
In my opinion, the benefits for end users are rather minimal since I doubt an average person would ever be checking C2PA provenance data, but there is a commercial incentive for Google and others to promote C2PA, since it makes preparing training material for Machine Learning significantly easier, and perhaps as a smaller benefit justifies hardware attestation that locks users down into proprietary OSes.
How?
Earlier quoted context omitted.
You're missing all of the points that there could be by focussing on random people. While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level. This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.
It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors. I like being contrarian as much as the next guy, but "Actuall…
And I agree on state actors. If a major one is invested in something like this they might have well compromised the signing project itself, the verification process, or even the court system or media. That seems like a rare and extremely high bar to guard against.
Earlier quoted context omitted.
In my opinion, the benefits for end users are rather minimal since I doubt an average person would ever be checking C2PA provenance data, but there is a commercial incentive for Google and others to promote C2PA, since it makes preparing training material for Machine Learning significantly easier, and perhaps as a smaller benefit justifies hardware attestation that locks users down into proprietary OSes.
> it makes preparing training material for Machine Learning significantly easier How?
Right now, the Internet training set is becoming more and more contaminated with better and better generative AI images and video.
It makes the models more screwed up, and makes it very difficult for humans to figure out what is original and not too.
If there was some signal that could at least make it easier to identify ‘original’/real images…
Earlier quoted context omitted.
> it makes preparing training material for Machine Learning significantly easier How?
A huge problem with ML training is the ‘ouroboros issue’ - training ML with input from other MLs breaks things in very deep (but difficult to stop/detect when it’s happening) ways due to the way the internal math works (model collapse). Right now, the Internet training set is becoming more and more contaminated with better and better generative AI images and video. It makes the models more screwed up, and makes it ve…
The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.