Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithm can pick out almost any American in supposedly anonymized databases

nytimes.com

Post reply on HN
Print view View original

91–100 of 101 posts

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#91

Post by Shendare »

SkyBelow wrote: post #82
mcintyre1994 wrote: post #68

Earlier quoted context omitted.

We covered this in a DB course with the term k-anonymity which seems to be standard in the literature, where a dataset is k-anonymous if every combination of characteristics (that can identify users) has at least K users. So in your case that dataset has only the 1-anonymity property, but you can set a k>1 and change the data set to satisfy it and improve the anonymity. Eg. if age was just stored as 90+ and there's a…

>I guess then the interesting question is how high does k have to be to call it anonymous vs pseudonymous. I think that for any size k less than the total size of the database, it is not anonymous. In cases like this, an overly strict definition favoring privacy is the only way to protect people. Similar to how we call 17 year olds children and treat them as such under law even though a 17 year old is far closer to a…

> I think that for any size k less than the total size of the database, it is not anonymous.

Wouldn't that require that every field of every record in the database be globally unique?

If something as simple as gender is a field in the database, the best k you could get would be the lowest count of records of each existent gender option.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#92

Post by SkyBelow »

Shendare wrote: post #91
SkyBelow wrote: post #82

Earlier quoted context omitted.

>I guess then the interesting question is how high does k have to be to call it anonymous vs pseudonymous. I think that for any size k less than the total size of the database, it is not anonymous. In cases like this, an overly strict definition favoring privacy is the only way to protect people. Similar to how we call 17 year olds children and treat them as such under law even though a 17 year old is far closer to a…

> I think that for any size k less than the total size of the database, it is not anonymous. Wouldn't that require that every field of every record in the database be globally unique? If something as simple as gender is a field in the database, the best k you could get would be the lowest count of records of each existent gender option.

It would mean that any identifying data being included would result in the data set not being considered anonymous. It will mean we have to tell people the data being sent to other organizations and companies is not anonymous, which is what should happen anyways. No more hiding behind 'we took the bare minimum steps required, if more advanced statistical information de-anonymized the data it isn't our fault'.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#93

Post by mpa000 »

al2o3cr wrote: post #30
stiffCompetitor wrote: post #9

I'm giving less and less of a fuck about this shit. Mostly because there's a point at which you refuse to be put on the run and don't want to hide. So maybe this is where the fight begins. At some point people make up their mind about dying not being a very big deal, and to become masters of their fate. To die on your feet, looking your accusers in the eye. Who would be so bold as to accuse me? Could I dare entertain…

Sir, this is an Arby's

Sir, this is the "Shockwave Rider" Readers Convention.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#94

Post by dwohnitmok »

SkyBelow wrote: post #82
mcintyre1994 wrote: post #68

Earlier quoted context omitted.

We covered this in a DB course with the term k-anonymity which seems to be standard in the literature, where a dataset is k-anonymous if every combination of characteristics (that can identify users) has at least K users. So in your case that dataset has only the 1-anonymity property, but you can set a k>1 and change the data set to satisfy it and improve the anonymity. Eg. if age was just stored as 90+ and there's a…

>I guess then the interesting question is how high does k have to be to call it anonymous vs pseudonymous. I think that for any size k less than the total size of the database, it is not anonymous. In cases like this, an overly strict definition favoring privacy is the only way to protect people. Similar to how we call 17 year olds children and treat them as such under law even though a 17 year old is far closer to a…

Is there any attribute of a person for which this is true?

I think what you're asking for is that any piece of data stored about someone be extensionally equivalent to "this is a human being" and no more which is not very useful (in an information-theoretic sense it has exactly zero use).

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#95

Post by dwild »

lmkg wrote: post #76
rzwitserloot wrote: post #10

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

> My go-to example to explain this is very simple: Let's > say we reduce birthdate info to just your birthyear, > and geoloc info to just a wide area. And then I have > an pseudonimized individual who is marked down as > being 105 years old. > Usually there's only one such person. I was interested to find that HIPAA's requirements for de-identification address the two particular issues you pointed out. First, age abo…

Theses are pretty good ways to anonymize theses fields, but they are still just good if only that field is used.

Let say that you got 20,000 inhabitants, you'll only need about 14 fields that are binary, much less fields if they are not binary (which is quite likely to happen). You'll most likely already got the gender... Even if you limit the age to 10 possible values, that's equivalent to 3 binary fields!

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#96

Post by Zenst »

shusson wrote: post #78
Zenst wrote: post #72

Earlier quoted context omitted.

I'd say that is pretty clear-cut with "making it impossible to derive insights on a discreet individual" If it is possible, it's not anonymous per GDPR's definition and that is what counts.

> "making it impossible to derive insights on a discreet individual" doesn't clarify how much information you already have about the individual. There is a distinction between being able to identify someone without any prior knowledge about them vs re-identifying them. I don't think the GDPR is clear about that.

Interesting - would an example of what you outline be say a digital voice recording. Which, unless you know who is in the recording, you have no way to associate that digital data with an individual.

Would that example fall within the remit you outline and as such - skirt the whole GDPR aspect?

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#97

Post by MaxBarraclough »

tialaramex wrote: post #5

If people tell you they're collecting data for statistical purposes, then one of three things: 1. They should deliberately introduce noise into the raw data. Nazis with the raw census data can spend all month trying to find the two 40-something Jews that data says live on this island of 8400 people, but they were just noise. Or were they? No way to know. 2. Bucket everything and discard all raw data immediately. This…

To points 1 and 2: It's proven very difficult to sanitize datasets in a way that ensures anonymity, but doesn't render it useless. You aren't the first to think of these kinds of transformations.

There are problems with Point 3: we're continually surprised with how effectively smart people can identify people in datasets expected to be 'safe'. You've also not accounted for that a collection of non-identifying attributes may become identifying.

That said, the GDPR is largely about prohibiting unnecessary data collection, in the spirit of Point 3. Hopefully it'll help at least a little.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#98

Post by m463 »

shalmanese wrote: post #60

While things like this sound scary, deanonymization turns out to be not exceedingly impactful in practice. Most entities have no desire to deanonymize you, to them, the most useful format is to treat you as a GUID with a bag of attributes attached to it. Of the entities that remain, they fall into two buckets: Ones powerful enough that they already have personally identifiable data without the need to deanonymize ano…

The amount of data and precision of web searches has grown enormously since the internet became active.

So, I would imagine everyone becomes more powerful over time.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#99

Post by Communitivity »

mcintyre1994 wrote: post #68
rzwitserloot wrote: post #10

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

We covered this in a DB course with the term k-anonymity which seems to be standard in the literature, where a dataset is k-anonymous if every combination of characteristics (that can identify users) has at least K users. So in your case that dataset has only the 1-anonymity property, but you can set a k>1 and change the data set to satisfy it and improve the anonymity. Eg. if age was just stored as 90+ and there's a…

Part of the problem here too is that we may be dealing with more than one data set. If there is enough information overlap in those sets then you could map a record from one set to another. In that situation you'd have more data about each person, reducing the k-anonymity of the combined data sets to below the k-anonymity of either individual data set.

That concerns me most around places that process data for other companies (e.g., Cambridge Analytics, Facebook, Google, Amazon). These places could have access to many different data sets relating to a person, and could potentially combine these data sets to uniquely identify a single individual.

I recently looked at something that I gave a fake zip, birth date, and gender. Based on statistical probabilities it gave a 68% chance of a large data set having 1-anonymity. Wasn't clear what they were considering large, so could be bogus, but if true imagine what could easily be done with 10+ unique fields (e.g., zip, birthdate, gender, married?, # of children, ever smoked?, deductible amount, diabetes?, profession, BMI).

The earlier poster is right, only aggregate data is truly anonymous.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#100

Post by doomjunky »

heavenlyblue wrote: post #87
rzwitserloot wrote: post #10

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

>> Aggregated data that can no longer be reduced to a single individual at all. Even aggregated data will loose the anonymisation characteristics when we are speaking of low volumes of data. Number of cancer patients in the area A: 1 Number of residents in the area A: 1

Even for big numbers, if x = 100% or x = 0%
Post reply on HN