21–30 of 77 posts
It seems that this won't work so great if you don't have any initial data to bootstrap yourself with. So, perhaps the idea is you bootstrap with a few people, do your explorations, and then scale it out with federation?
How do they assure you that the training algorithm isn't just exfiltrating your data? Edit: By that I mean... What's stopping the model from being as simple as "learn my personal information"?
It's unclear from the comic how the masks are coordinated, or how they compensate for the risk that a participating device drops out (which will make all the other data from that iteration useless, if you set the masks up in a naive way.)
All right, I'm cynical as all heck about ad companies and privacy, but this has me optimistic. Somebody disillusion me, why shouldn't I be optimistic?
Of course it depends on the application... For example, I might not want my keyboard autocomplete learning from others, but I might want my self driving car to do it.
For applications where we want a common model, I see no other way to do it but this. The idea companies collect massive stockpiles of data forever is infeasible.
What happens with the zero-sum cancelling out phase if one device disappears during the process?
How do they assure you that the training algorithm isn't just exfiltrating your data? Edit: By that I mean... What's stopping the model from being as simple as "learn my personal information"?
It looks from the comic like they aggregate data with linear structure (probably derivatives on the model parameters.) Each device adds a mask to their part of the data, and somehow the masks are coordinated across devices so that when the data are summed on the central training server, the masks cancel out. It's unclear from the comic how the masks are coordinated, or how they compensate for the risk that a particip…
Thanks, that's the part I must have glossed over. It looks like they're using secret sharing to distribute as shares that all need to be together to reassemble [1].
[1] https://storage.googleapis.com/pub-tools-public-publication-...
All right, I'm cynical as all heck about ad companies and privacy, but this has me optimistic. Somebody disillusion me, why shouldn't I be optimistic?
Well the cynical view would be. 1) This still lets you have personalized models, just trained on more than 1 user, thats fine at google's scale anyway 2) Their competitors (FB, AMZN) dont have the edge compute (Android) to do this, and to a lesser degree don't have the ML stack (however Android implements this at the API level will be very Tensorflow focused) 3) Now google can push for privacy regulations that preven…
How do they assure you that the training algorithm isn't just exfiltrating your data? Edit: By that I mean... What's stopping the model from being as simple as "learn my personal information"?
It looks from the comic like they aggregate data with linear structure (probably derivatives on the model parameters.) Each device adds a mask to their part of the data, and somehow the masks are coordinated across devices so that when the data are summed on the central training server, the masks cancel out. It's unclear from the comic how the masks are coordinated, or how they compensate for the risk that a particip…
All right, I'm cynical as all heck about ad companies and privacy, but this has me optimistic. Somebody disillusion me, why shouldn't I be optimistic?
In fact, if this is seriously gonna be used as a "better privacy" argument (as some people seem to be already doing in this very thread), I'm calling it a PR victory for the "bad guys".
First off, if you were worried about what Android was sending to Google, there's no reason to believe it's going to stop. In fact, I believe that the biggest problem always was the (carefully cultivated) confusion about what data is being sent: you have a hundred of menus to "opt-out" of something and it's not even exactly clear if it changes anything. In fact, we know for a fact, that when "opting out" in some cases more data is being sent. And even if we are not entirely happy about it, most of us still allow this to happen, because there's nothing we can directly do to prevent it and everybody says "well, I do need a smartphone after all, right?" (yeah-yeah, somebody doesn't, but we are not talking about the weird minority here)
And all of it happens when it's relatively straightforward to see what data is being sent, because of minimal aggregation on the device. And still even somewhat technically-minded people don't really know what Google (Facebook, Amazon, whatever) really knows about them.
Second, what really is "federated learning"? Well, let's imagine no humans speak Chinese, but there is this program (owned by Google), that does. And speaking Chinese is how it actually operates internally, when deciding to show this or that ad to you, or sending a ballistic missile to your location. So, in order for it to learn, we normally were sending English sentences, which were translated to Chinese server-side. Federated learning is when they are translated to Chinese client-side (which might be considered "lossy" conversion, but to what degree is not really specified), and then sent to Google to be aggregated.
So, yeah, no raw data has been sent, but the central Chinese-speaking machine still somehow knows it all. What exactly it knows, depends on what we really meant by "translating into Chinese" in our metaphor. But effectively, we just offloaded some processor work to the client side, which, as I said, seems really cool to me from the technical perspective, but there's no way it automatically protects us from anything.
Third is basically 1 + 2: we didn't know what is being sent when it was all raw-data, we will know even less, when it's client-side aggregated in some unintelligible-for-the-humans way. And it scares me even more, because if it allows some PR victories for the Google&Friends, then sky is the limit for what more surveillance can be done this way. I mean, if it would be publicly known that Android sends all the sound and all the image from your mic & camera to Google, I think (I hope!) that people would seriously oppose to that. But if it's not the real images, but just some matrix of weights, learnt from them — it might be less clear if anybody has to object to that. And I think they absolutely have to! Because if we don't make any very restrictive assumptions about what we mean by "learning" in this very specific case, then the only thing that matters is that the "central brain" still saw all these images, it just isn't known what exactly it "remembered".
After all, we, humans, also don't store all the pictures we've seen in our brains: it doesn't make you much happier if I saw your transaction history (or whatever else you don't want me to know), because it was never the picture I was after, but only the "aggregated info".
Earlier quoted context omitted.
It looks from the comic like they aggregate data with linear structure (probably derivatives on the model parameters.) Each device adds a mask to their part of the data, and somehow the masks are coordinated across devices so that when the data are summed on the central training server, the masks cancel out. It's unclear from the comic how the masks are coordinated, or how they compensate for the risk that a particip…
> It looks from the comic like they aggregate data with linear structure (probably derivatives on the model parameters.) Thanks, that's the part I must have glossed over. It looks like they're using secret sharing to distribute as shares that all need to be together to reassemble [1]. [1] https://storage.googleapis.com/pub-tools-public-publication-...
From the introduction, it looks like they group participants into smaller clusters, coordinate between those via the centralized server in a star topology, use Diffie-Helman to share secrets between the participants in a cluster, and construct the canceling noise within that cluster.
The Shamir secret sharing squicks me a bit. It looks like if the adversary can control cluster membership (and Google is the adversary, here), they can recover the gradients.
> To prevent the server from simulating an arbitrary number of clients (in the active-adversary model), we require the support of a public key infrastructure that allows clients to register identities, and sign messages using their identity, such that other clients can verify this signature, but cannot impersonate them. In this model, each party u will register to a public bulletin board during the setup phase. The bulletin board will only allow parties to register keys for themselves, so it will not be possible for the attacking parties to impersonate honest parties.