Federated Learning

Differential privacy is exactly meant for this, in fact. Differential privacy adds a certain amount of randomly-generated noise to client inputs. The result is that, statistically speaking, it’s impossible to tell the difference between a model with your data in it and a model without your data in it.

Arguably the reason the comic doesn’t mention differential privacy is that it’s neither new nor invented at Google. Or maybe just because it’s not technically part of federated learning. But the “federated learning at scale” paper Google put out mentions it, and says they have implemented DP techniques.

For example, say we deployed federated learning to train a predictive language model, and allowed it to learn from emails, say, inside Google. Looking at what the model predicts when you type "Here at Google our next secret project is..." could very likely reveal something they wouldn't want widely revealed.

I wouldn't be critiquing your response if you had something a little less handwavey to say. When people point out flaws in protocols, I like to see specific exploit examples, like the kind you'd put into a Spectre/Meltdown Advisory.

If you said "out of order speculative execution in CPUs might eventually allow exploits", I may even have vaguely agreed, but without a concrete criticism, it's more of an "uneasy feeling" you have.

I can make loose arguments too. Everything you do in this word leaks entropy. Your information is entangled with other people, leaving a wake behind you from the moment you're born. There's a gazillion side channels hanging off of you. So learning information about you is pretty much a given. The question is, is it relevant or important information? There's a huge difference between "learning something", "learning something sensitive about me (that I share with a large number of other people)" and "learning something sensitive about me that's individually traceable to me"

Most people won't care if a Federated Learning model, using data from your phone, learns that people who stay up late, and search for Coke, also end up with diabetes, anymore than a double blind study learns about the risks of smoking and lung cancer -- the doctors have learned information about you (you're a smoker, and you have/don't have lung cancer), but they haven't learned that you, krick, are a smoker.

The whole point of differential privacy, federated learning, and other techniques, is that aggregate statistics, and aggregate models can be learned without any personally traceable information.

Now, you could argue that somewhere, deep within the logical depth of the weights of a DNN is some kind of personal information that could be deanonymized, but this is like a claim that you found a weakness in a hash function -- until you show it, it's just a claim, and mathematics and security research is full of wrong claims on both sides.