Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using ‘radioactive data’ to detect if a data set was used for training

ai.facebook.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Using ‘radioactive data’ to detect if a data set was used for training

#11

Post by lmkg »

rustyconover wrote: post #10
ISL wrote: post #7

It is hard to believe that modifying input datasets won't modify the qualitative behavior of the outputs in some way. This appears to be a modern variation of the https://en.wikipedia.org/wiki/Fictitious_entry / copy-trap behavior that mapmakers have made in the past.

I think most ML models aren’t very “lean”, meaning there is space in their weight layers for information isn’t directly attributable to predictive accuracy. That space is likely where this new “radioactive” like data is being “stored”/“remembered”. The leanness could be increased during training by progressively trimming width/depth of weights, but I doubt if every model has this done.

This is definitely true. In fact, this can be exploited to extract sensitive/private attributes about the training data from the learned models. This may become an issue for, e.g., AI in healthcare.

"Overlearning Reveals Sensitive Attributes": https://arxiv.org/abs/1905.11742

Re: Using ‘radioactive data’ to detect if a data set was used for training

#13

Post by bordercases »

SeriousM wrote: post #8

> Radioactive data could also help protect against the misuse of particular data sets in machine learning. This last sentence is the real reason behind this technology. Training data isn't cheap and I'm sure the paying party needs a watermark on it.

"Watermarking" and trademarking can be different things. And access to data is already licensed.

I think you're right in that DRM systems are likely to be built on top of such infrastructure, but DRM has been broken in other contexts before and the system doesn't necessarily have to be used for DRM.

Re: Using ‘radioactive data’ to detect if a data set was used for training

#14

Post by alcinos »

c1ccccc1 wrote: post #5

I'm surprised that it's even necessary to modify the dataset to achieve this. From what I've read, large models will often memorize their training data, and it seems like even with smaller models it should be possible to tell whether or not it was trained with some set of images, simply because the loss will be lower.

It is already possible to know if a particular image has been used in training (see eg. https://arxiv.org/abs/1809.06396 by the same authors), but this new work also provides a p-value to give you a confidence on the result it gives.

Also notice that being proactive in watermarking the dataset can be desirable in some cases. For example, many datasets have large overlaps in the base images they use (but sometimes different labels), so it can be interesting to know whether a model was trained on "your" version of the dataset.

Re: Using ‘radioactive data’ to detect if a data set was used for training

#16

Post by zzbn00 »

Not relevant to the main trust of the article but barium sulphate is not radioactive, it just efficiently absorbs X-rays. Radioactive markers are I believe most commonly used in PET scans, Wikipedia suggests flourine-18 as the common isotope used.

Re: Using ‘radioactive data’ to detect if a data set was used for training

#17

Post by mring33621 »

Have not yet read the article, as Facebook is blocked at work, but I would guess that this is mostly the application of steganographic techniques, to hide known patterns, in datasets that are likely to be stolen/borrowed for training.

Then observe the outputs of said models to try to discern related patterns.

Re: Using ‘radioactive data’ to detect if a data set was used for training

#18

Post by SkyBelow »

sillysaurusx wrote: post #3

At first glance, this seems like one of the more interesting projects to come out of Facebook AI. Justification: In the future, AI models will increasingly become interwoven with tech. It's not going to be so much "AI programming" as just "programming". That raises an interesting question – one that has bothered me for a long time: Who owns copyright on training data? As we saw with Clearview AI, a lot of data is bei…

Is there any particular reason to think this won't become another cat and mouse escalation as training algorithms have built in protection against this (and other related training set manipulations, especially the poisoning one the article talked about)? That isn't to say it is useless, as most cat and mouse escalations prove to be quite useful as long as the mouse stays a little ahead of the cat.

In this case, wouldn't such a marker be able to be detected by looking at images of the same class and seeing if there are any common perturbation across them, adjusting the images by the common perturbation , and then training the neural network? Even if there isn't such a common perturbation across them, adjusting them by the false flag common perturbation generated shouldn't be any more destructive than this method would be.

If there was a way to make it dependent upon the initial image and the class, that would be much harder to detect, but would such a method be possible to detect since all images within a class would not have the common perturbation?

Re: Using ‘radioactive data’ to detect if a data set was used for training

#19

Post by gambler »

sillysaurusx wrote: post #3

At first glance, this seems like one of the more interesting projects to come out of Facebook AI. Justification: In the future, AI models will increasingly become interwoven with tech. It's not going to be so much "AI programming" as just "programming". That raises an interesting question – one that has bothered me for a long time: Who owns copyright on training data? As we saw with Clearview AI, a lot of data is bei…

>Who owns copyright on training data?

Megacorps. Regardless of what the data is, who produced that data or when.

Re: Using ‘radioactive data’ to detect if a data set was used for training

#20

Post by eganist »

Not mentioned thus far anywhere in the article or in comments: potentially weaponizing this against deep fakes.

What's to stop cameras from making raw photos "radioactive" from now on, making deepfakes traceable by tainting the image-sets on which the models generating the deepfakes were trained?

This isn't my field. I'm certain there's a workaround, but I'd suspect detecting sufficiently well-placed markers would require knowing the original data pre-mark, which should be impossible if the data is marked before it's written to camera storage. I haven't even fully thought out the logistics yet, such as how to identify the radioactive data.

But am I missing something? I feel like this is viable.

Post reply on HN