Live data from Hacker News

Algorithm can pick out almost any American in supposedly anonymized databases

nytimes.com

11–20 of 101 posts

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#12
post #8

If people tell you they're collecting data for statistical purposes, then one of three things: 1. They should deliberately introduce noise into the raw data. Nazis with the raw census data can spend all month trying to find the two 40-something Jews that data says live on this island of 8400 people, but they were just noise. Or were they? No way to know. 2. Bucket everything and discard all raw data immediately. This…

Nazis used the census because it was there. Were it not, they would have just went house to house - which they then regularly did anyway. The antidote of oppression is not the Index statisticum prohibitorum, but quite the opposite, education, and in particular educating about how different each and every one of us is, and yet it doesn't take much to get along well.

Today's Nazis would probably use machine learning to identify "characteristically Jewish facial features".

(Or, of course, simply pass a law that entitles them to deport or hold indefinitely anyone who can't prove they're a real Ayrian with only the papers they have on them at the time)

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#13
In any database whose fields have at least 4 meaningful ranges, 16 fields give 4.000.000.000 possibilities. Now I am on my iphone. As long as ranges are meaningful (i.e. they do divide the group in somewhat even parts), the individuating possibilities are HUGE. And fields do have usually many more than 4 ranges.

Anonymizing datasets is a weasel term.

The database is secure or it is not. As any database is quite likely insecure, we are doomed.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#14

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

It doesn't roll off the tongue, perhaps pseudo-anonymization is enough.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#15

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

When I worked in the data analysis world, we would have a bounds for this stuff.

Number of people >100 in the area: <10

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#16
When information you are recieving has been "anonymized", how do you tell it is accurate? If you are the person collecting and storing said sensative information, you will know what is and is not accurate, but nobody else knows.

For some kinds of information, like medical records, the information is deadly not to have accurate, but also deadly to have accurate and public. Once the information leaks, employers might decide to not hire high-risk people or insurers might decide to pass over certain people as too costly.

I'm of the opinion "anonymizing" data is something that enables grifting; if enables the collectors to placate the people they are pulling data from, and it allows the grifters to make the argument the information they have means nothing.

Ultimately, I think these organizations should be making sure their information is absolutely accurate, and we should have laws in place, with severe criminal penalties, against the use transfer or use of said information. I would even go so far as to say things like cell phone location records should be fully public as a matter of the law.

Now when you want to get those records, you go to a government website for the "hunt and poke" stuff (e.g. where are my kids going or is my wife spending time with another lover, how long is my commute on average, or where was I at 3 years ago on a day, all sorts of useful questions); the access records are public too.

If you want to study them, you sign a NDA saying you won't, under penalty of severe criminal prosecution, leak the information or use it for criminal purposes. Anyone found having the data and no signed government NDA = instant 20 year prison sentance plus felony conviction.

This way, if, for example, someone signs the NDA and goes on to offer services to executives to help them cherry pick staff, not only does the person offering the under the table services go to jail, but the executive does as well.

When you criminalize certain things, then give the public all the information and tools to do as they see fit, the law works. It's a lot easier to prosecute a company executive for cherry-picking staff with insurance data when the data is well-labeled. It is also a lot easier to sue them when you have an access record that says someone under their employ checked how often you go to a clinic or night club via your cellphone records.

The problem is not going away anyway, and "anonymizing" data to placate our sense of morality isn't going to help. There is no easy technical solution, but if the thinking is not to anonymize but instead track and enforce who has access, things change drastically.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#17

I'm a programmer in the GP data analysis world. We use the term 'pseudonymization' for this kind of data. 'Anonymization' is used solely to refer to, say, 'the sum total of diabetes patients this practice has' (that would be anonymous patient data; it would not be anonymous relative to the GP office this refers to): Aggregated data that can no longer be reduced to a single individual at all. The term raises questions…

I worked on a reporting/dataviz project a few years back, showing survey results from states/districts/schools. All the data was included in the totals, but if a school had 5 or fewer responses, we didn't allow viewing of the school data. It was specifically because being able to see answers to things like "do you support your principal?" when there were only, say, 3 teachers at a school, and seeing 2 for "no"... it was way too easy to determine who the respondents were. Even with '5' as a cutoff, it still felt a bit dicey for some schools.

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#18
post #6

The article title misses a bit of nuance from the paper which is specifically talking about re-identification. e.g from the paper: "We show that, as a male born on July 31, 1945 and living in Cambridge (02138), the information used by Latanya Sweeney at the time, William Weld was unique with a 58% likelihood (ξx = 0.58 and κx = 0.77), meaning that Latanya Sweeney’s re-identification had 77% chances of being correct.…

What accounts for that remaining 2‰ of uncertainty?

Re: Algorithm can pick out almost any American in supposedly anonymized databases

#19
post #11

where's the link to code?

The paper suggests it can be accessed on the site [0], however certain parts of the site only appear if you run through their questionnaires.

> The source code to reproduce the experiments is available at https://cpg.doc.ic.ac.uk/individual-risk, along with documentation, tests, and examples.

As far as I can tell, the source code is not available, at least not from where the authors suggest.

[0] https://cpg.doc.ic.ac.uk/individual-risk/

Post reply on HN