Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is Differential Privacy?

blog.cryptographyengineering.com

Post reply on HN
Print view View original

51–60 of 108 posts

Re: What is Differential Privacy?

#51

Post by Twisell »

singularity2001 wrote: post #37
Twisell wrote: post #26

Earlier quoted context omitted.

This is so wrong. The only thing that isn't user-encrypted so far is the iPhone backup on their server (Of course it is encrypted but Apple have the key to decrypt it as needed). The official explanation so far is that if the user forgot the password a user-encrypted backup would just become some useless junk. This is (officially) the sole remaining non user-encrypted personal data on apple server that authority can…

The burden of proof should be on Apple's site. Given their secrecy all you can do is pray or switch to open source.

Alternatively you can also RTFM before complaining about secrecy... https://www.apple.com/business/docs/iOS_Security_Guide.pdf

Of course that will never be as audit friendly as an Open-Source code. But don't call it a secret, while you actually just didn't search for the information...

Re: What is Differential Privacy?

#52

Post by fauigerzigerk »

johncolanduoni wrote: post #12
quantumtremor wrote: post #11

Earlier quoted context omitted.

At some point, regardless of adding noise, you're definitely losing your privacy. I'd be happy with an "opt-out" feature that I know worked (as far as I can see, only if it was open-source). I didn't watch WWDC, perhaps they mentioned this.

I agree opt-out is definitely something that should be deployed alongside differential privacy, but what makes you so sure that it doesn't work "at some point"? If the noise means an specific query to one user's information has a significant chance of being wrong, how does this not equate to privacy? You can add a lot more noise than you might imagine if you know the kind of analysis you'll be doing with the data; fo…

The whole point of collecting the data is to predict the actions or information needs of individual users. That in itself is a privacy issue.

If a recommender system for iTunes can predict the likelihood of me appreciating movies that contain violence against women, that information could be subpoenaed when I am falsely accused of having strangled my girlfriend.

I appreciate that Apple is trying to protect our privacy where they can. But if we want them to make predictions about or behavior, we have to be aware of the fact that we are necessarily giving up some privacy.

Re: What is Differential Privacy?

#53

Post by Create »

Create wrote: post #38
cyphar wrote: post #10

Earlier quoted context omitted.

The point of differential privacy is to allow for aggregate analysis, without destroying the privacy of outliers. Researchers deal with noise all the time, so is it so odd that a field of researchers believe that adding enough noise to data released with studies will allow for conclusive analysis without ruining privacy for individuals?

Outliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not F…

downvoter: care to elaborate on the usefulness of a database with almost identical entries with MEANingless values?

Re: What is Differential Privacy?

#54

Post by corv »

Twisell wrote: post #26
conradev wrote: post #22

Earlier quoted context omitted.

Even though the technique doesn't require collecting all of the data, Apple still does collect a huge well of data. They store all iCloud sync material (backups, photos, contacts, calendars, mail, documents, etc.) without end to end encryption, and have all of the iMessage metadata.

This is so wrong. The only thing that isn't user-encrypted so far is the iPhone backup on their server (Of course it is encrypted but Apple have the key to decrypt it as needed). The official explanation so far is that if the user forgot the password a user-encrypted backup would just become some useless junk. This is (officially) the sole remaining non user-encrypted personal data on apple server that authority can…

Are you saying Notes, Safari Bookmarks, Photos, etc are encrypted on iCloud?

How come they are accessible from iCloud.com? Decrypted by the browser on the fly?

Re: What is Differential Privacy?

#55

Post by johncolanduoni »

fauigerzigerk wrote: post #52
johncolanduoni wrote: post #12

Earlier quoted context omitted.

I agree opt-out is definitely something that should be deployed alongside differential privacy, but what makes you so sure that it doesn't work "at some point"? If the noise means an specific query to one user's information has a significant chance of being wrong, how does this not equate to privacy? You can add a lot more noise than you might imagine if you know the kind of analysis you'll be doing with the data; fo…

The whole point of collecting the data is to predict the actions or information needs of individual users. That in itself is a privacy issue. If a recommender system for iTunes can predict the likelihood of me appreciating movies that contain violence against women, that information could be subpoenaed when I am falsely accused of having strangled my girlfriend. I appreciate that Apple is trying to protect our privac…

You're misunderstanding where this is to be used. It is specifically not for things like iTunes suggestions, where it would be useless. It's for situations where they want to get aggregated metrics without collecting identifiable information. The obfuscation can be performed by the client so that they never have a database on the server with accurate (at the specific user level) data.

Re: What is Differential Privacy?

#56

Post by cyphar »

Create wrote: post #38
cyphar wrote: post #10

Earlier quoted context omitted.

The point of differential privacy is to allow for aggregate analysis, without destroying the privacy of outliers. Researchers deal with noise all the time, so is it so odd that a field of researchers believe that adding enough noise to data released with studies will allow for conclusive analysis without ruining privacy for individuals?

Outliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not F…

> Outliers are the reason databases exist.

Disagree. Data is why databases exist.

> Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis.

I said "aggregate", not "average". There are many kinds of aggregate analysis useful (in Astrophysics, you can take many different samples from different stars and use the aggregate to compute commonalities in the sample that you would've detect with a single measurement). There is more to aggregate analysis than averaging data.

As for the rest of your points, I'm not a statistician so I can't comment. Also, I didn't downvote you (HN rules).

Re: What is Differential Privacy?

#57

Post by cyphar »

Create wrote: post #53
Create wrote: post #38

Earlier quoted context omitted.

Outliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not F…

downvoter: care to elaborate on the usefulness of a database with almost identical entries with MEANingless values?

Outliers are not the only useful thing in a set of data. If you remove outliers from most data sets, it doesn't suddenly become "almost identical" -- unless your outlier rejection system is "is it equal to 1".

Re: What is Differential Privacy?

#58

Post by fauigerzigerk »

johncolanduoni wrote: post #55
fauigerzigerk wrote: post #52

Earlier quoted context omitted.

The whole point of collecting the data is to predict the actions or information needs of individual users. That in itself is a privacy issue. If a recommender system for iTunes can predict the likelihood of me appreciating movies that contain violence against women, that information could be subpoenaed when I am falsely accused of having strangled my girlfriend. I appreciate that Apple is trying to protect our privac…

You're misunderstanding where this is to be used. It is specifically not for things like iTunes suggestions, where it would be useless. It's for situations where they want to get aggregated metrics without collecting identifiable information. The obfuscation can be performed by the client so that they never have a database on the server with accurate (at the specific user level) data.

I don't think I am misunderstanding (although I'm not completely sure about that). My point isn't about iTunes. My point is about the purpose of data collection. If that purpose is predicting our actions, then that in itself is a privacy issue.

I understand that the database Apple wants to build does not contain accurate information about individual users. But if that database allows them to make predictions of our behavior, then there is a privacy issue. If the purpose is not prediction, then what is it?

Re: What is Differential Privacy?

#59

Post by gear54rus »

liquidise wrote: post #19
kordless wrote: post #17

Earlier quoted context omitted.

Fuck Apple. The only thing they are programmed to care about is the bottom line and getting more users. That programming is complicated and unlikely to change to suit the consumer's needs, or desires.

You say that despite this privacy push being a direct response to consumer sentiments. Apple knows well that as tech becomes wearables and the Internet of things, privacy concerns skyrocket. My grandparents won't buy things online. Soon my generation will be those old and anxious curmudgeons unless our concerns are eased. Do they care about their bottom line? Of course. It's for that very reason they are investing th…

This 'push' is just another unprovable unverifiable marketing bs that is just enough for the type of audience they attract until it's all opened to FS licenses and hardware is made to accept any software that user wants it to (like it should if you bought it).

Re: What is Differential Privacy?

#60

Post by peteretep »

kbenson wrote: post #41
anshukla wrote: post #35

Earlier quoted context omitted.

How do you explain their insistence, then, on doing object recognition in photos on-device? Is it possible that they have understood the affordances of various data collection and obfuscation techniques and will apply the appropriate ones after taking into account their desire to protect privacy? Google indeed has RAPPOR (and other projects, I'm sure), but the cultural difference Apple claims is "we consider privacy…

> the cultural difference Apple claims is "we consider privacy in everything we do" I'm pretty sure that should be interpreted as "we've determined privacy is a differentiator in the market, so as of some indeterminate time in the past, ranging from a few years ago to our inception, we consider privacy in everything we do." Now, there's nothing wrong with that, and that's not to say they haven't been privacy consciou…

If Tim Cook wasn't part of a frequently and historically persecuted minority, I'd be more cynical too.
Post reply on HN