Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is Differential Privacy?

blog.cryptographyengineering.com

Post reply on HN
Print view View original

61–70 of 108 posts

Re: What is Differential Privacy?

#61

Post by Create »

cyphar wrote: post #56
Create wrote: post #38

Earlier quoted context omitted.

Outliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not F…

> Outliers are the reason databases exist. Disagree. Data is why databases exist. > Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. I said "aggregate", not "average". There are many kinds of aggregate analysis useful (in Astrophysics, you can take many different samples from different stars and use the aggregate to compute commonalities in the sample that you would've det…

sorry -- please substitute "average" (original was also in quotes) to category or factor, and you still have the bin I am talking about. You can put any label on it you like such as "commonality", as long as you remove details, ie. other bins.

But as you say: your "aggregate analysis" NEEDS "many different samples from different stars". Commonality is the result of your analysis based on different samples. But since they are common, you can go and sample and have the result without doing mass surveillance on every star.

ps: I am fully aware of photo stacking, but also note, that stars are not humans, see context of privacy. Please look at argus or sdcMicroGUI from CRAN to get a feeling for data utility vs. reidentification risk.

Re: What is Differential Privacy?

#62

Post by [deleted] »

cyphar wrote: post #56
Create wrote: post #38

Earlier quoted context omitted.

Outliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not F…

> Outliers are the reason databases exist. Disagree. Data is why databases exist. > Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. I said "aggregate", not "average". There are many kinds of aggregate analysis useful (in Astrophysics, you can take many different samples from different stars and use the aggregate to compute commonalities in the sample that you would've det…

[deleted]

Re: What is Differential Privacy?

#63

Post by johncolanduoni »

fauigerzigerk wrote: post #58
johncolanduoni wrote: post #55

Earlier quoted context omitted.

You're misunderstanding where this is to be used. It is specifically not for things like iTunes suggestions, where it would be useless. It's for situations where they want to get aggregated metrics without collecting identifiable information. The obfuscation can be performed by the client so that they never have a database on the server with accurate (at the specific user level) data.

I don't think I am misunderstanding (although I'm not completely sure about that). My point isn't about iTunes. My point is about the purpose of data collection. If that purpose is predicting our actions, then that in itself is a privacy issue. I understand that the database Apple wants to build does not contain accurate information about individual users. But if that database allows them to make predictions of our b…

It could be a number of things, but one possibility is identifying broad correlations between metrics. Ssince you can't trust the accuracy of the individual metrics, you will have a limited ability to apply the correlation to individual users, but if you use the right kind of noise aggregated conditional probabilities may survive.

So Apple can (for example) predict that listing to band A means you are likely to like band C, and then send a list of correlations to your device so the predictions can be made there by examining your library locally. A more probable use is analytics for marketing purposes. Another is selling just these correlations and other aggregate statistics to other parties; this is actually how Mint makes money.

Re: What is Differential Privacy?

#64

Post by Twisell »

corv wrote: post #54
Twisell wrote: post #26

Earlier quoted context omitted.

This is so wrong. The only thing that isn't user-encrypted so far is the iPhone backup on their server (Of course it is encrypted but Apple have the key to decrypt it as needed). The official explanation so far is that if the user forgot the password a user-encrypted backup would just become some useless junk. This is (officially) the sole remaining non user-encrypted personal data on apple server that authority can…

Are you saying Notes, Safari Bookmarks, Photos, etc are encrypted on iCloud? How come they are accessible from iCloud.com? Decrypted by the browser on the fly?

It seems so: https://support.apple.com/en-us/HT202303

However I reckon that technically Apple could access data or give data stored on iCloud to NSA/FBI because they actually still hold the keys for that part too (not only backup as I thought). Only the password/creditcard Keychain is now claimed to be fully user-encrypted and can't be recovered by any mean by apple.

For anything else than a warrant, they'll "just" have to breach every engagement they made in their contract which would, as far as I know constitute a pretty solid legal case that could only lead a public walk of shame that could compromise the whole company's future.

If you don't trust them, don't use their cloud, I totally respect that. In the end it always appeal to some degree of trust, even GitHub could be spying on paid private repositories under the hood if they really wanted to. But for what gain?

Re: What is Differential Privacy?

#65

Post by fauigerzigerk »

johncolanduoni wrote: post #63
fauigerzigerk wrote: post #58

Earlier quoted context omitted.

I don't think I am misunderstanding (although I'm not completely sure about that). My point isn't about iTunes. My point is about the purpose of data collection. If that purpose is predicting our actions, then that in itself is a privacy issue. I understand that the database Apple wants to build does not contain accurate information about individual users. But if that database allows them to make predictions of our b…

It could be a number of things, but one possibility is identifying broad correlations between metrics. Ssince you can't trust the accuracy of the individual metrics, you will have a limited ability to apply the correlation to individual users, but if you use the right kind of noise aggregated conditional probabilities may survive. So Apple can (for example) predict that listing to band A means you are likely to like…

>So Apple can (for example) predict that listing to band A means you are likely to like band C

And how is that different from my iTunes example?

Re: What is Differential Privacy?

#66

Post by cyphar »

Create wrote: post #61
cyphar wrote: post #56

Earlier quoted context omitted.

> Outliers are the reason databases exist. Disagree. Data is why databases exist. > Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. I said "aggregate", not "average". There are many kinds of aggregate analysis useful (in Astrophysics, you can take many different samples from different stars and use the aggregate to compute commonalities in the sample that you would've det…

sorry -- please substitute "average" (original was also in quotes) to category or factor, and you still have the bin I am talking about. You can put any label on it you like such as "commonality", as long as you remove details, ie. other bins. But as you say: your "aggregate analysis" NEEDS "many different samples from different stars". Commonality is the result of your analysis based on different samples. But since…

> But since they are common, you can go and sample and have the result without doing mass surveillance on every star.

"Mass surveilance" reduces noise and lets you get more data in a shorter period of time (telescopes have large fields of view, but they can't make time pass faster). Stacking (which is what the technique is called in Astrophysics) is very useful in this case. Not to mention that you can also do individual analysis as well.

Actually, most interesting of all is that you can do this type of analysis on objects like neutron stars that we can't observe directly because they're too faint. Because noise in telescopes can be modelled as a Poisson process, stacking actually increases S/N in a way you can't do without making much bigger telescopes.

PS. I'm not a statistician, so I can only speak to what I know. But my whole point is that researchers do know how to deal with noisy data, regardless of whether or not that noise is man-made or not. Interestingly enough, I found out recently that the NASA pipeline actually breaks certain data sets they have released (which have papers written about them) so man-made noise is a problem regardless of whether or not it's intentional.

Re: What is Differential Privacy?

#67

Post by johncolanduoni »

fauigerzigerk wrote: post #65
johncolanduoni wrote: post #63

Earlier quoted context omitted.

It could be a number of things, but one possibility is identifying broad correlations between metrics. Ssince you can't trust the accuracy of the individual metrics, you will have a limited ability to apply the correlation to individual users, but if you use the right kind of noise aggregated conditional probabilities may survive. So Apple can (for example) predict that listing to band A means you are likely to like…

>So Apple can (for example) predict that listing to band A means you are likely to like band C And how is that different from my iTunes example?

I used "you" incorrectly, my bad. They can predict that people who listen to band A are likely to like band C, but their data for whether you listen to band A still has a significant chance of being wrong.

Re: What is Differential Privacy?

#68

Post by cromwellian »

Some dissenting views on the utility of differential privacy: https://medium.com/@Practical/differential-privacy-considere...

Also, Apple is woefully low on details, theoretical privacy should be accompanied by openly published research papers that are peer reviewed. I understand they won't release the source, but would you trust Apple if they said they invented a new encryption algorithm, but refuse to publish an academic paper on it? I'd be interested precisely in what they're doing. Are they claiming they're doing federated learning, by gathering anonymous image data from photos, uploading it to their cloud, training DNNs on it, and then shipping the results back down to clients for local recognition? Surely they're not training on device, as this is very RAM and CPU intensive.

Re: What is Differential Privacy?

#69

Post by dudusmaximus »

applester wrote: post #4
hfsbtnye wrote: post #2

I have to admit, I'm really starting to like the direction that Apple is heading despite being previously disenchanted. I only wish that they would go ahead and put everything under a free software license, since they're in the business of selling hardware that's coincidentally bundled with their software.

To paint another viewpoint, Apple initially went all gung ho about privacy and wanted to make not collecting data a big play (and fairly so, full respect to them). The recent WWDC obviously shows a big shift towards AI and ML applications within the company. Some things are possible on the device, but many neural nets just cannot be served from an iPhone reasonably. Hence, the move towards more data collection. I rea…

Differential Privacy is now $AAPL's licence to collect more "anonymised" personal data & benefit. Such personal data collection is not an evil anymore. I'll have some of the "PR" they're having!

Re: What is Differential Privacy?

#70

Post by HappyTypist »

Twisell wrote: post #64
corv wrote: post #54

Earlier quoted context omitted.

Are you saying Notes, Safari Bookmarks, Photos, etc are encrypted on iCloud? How come they are accessible from iCloud.com? Decrypted by the browser on the fly?

It seems so: https://support.apple.com/en-us/HT202303 However I reckon that technically Apple could access data or give data stored on iCloud to NSA/FBI because they actually still hold the keys for that part too (not only backup as I thought). Only the password/creditcard Keychain is now claimed to be fully user-encrypted and can't be recovered by any mean by apple. For anything else than a warrant, they'll "just" h…

CISPA grants civil immunity for sharing information with the government.
Post reply on HN