Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

latanyasweeney.org

Post reply on HN
Print view View original

61–70 of 102 posts

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#61

Post by reinhardt »

reinhardt wrote: post #32
jdp23 wrote: post #5

Here's some backstory, from the 1990s when the Massachusetts' Group Insurance Commission released "anonymized" health data: ''At the time GIC released the data, William Weld, then Governor of Massachusetts, assured the public that GIC had protected patient privacy by deleting identifiers. In response, then-graduate student Sweeney started hunting for the Governor’s hospital records in the GIC data. She knew that Gove…

If only 6 people out of 54000 shared Weld's birthday, she was just lucky (not that this invalidates the main point).

Edit: Ok ok got it, birthday includes the year, my mistake; no need to keep downvoting

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#63

Post by cpeterso »

kingkilr wrote: post #27
xxbondsxx wrote: post #23

Earlier quoted context omitted.

Birth location =/= mailing address zipcode. Babies are usually born in hospitals, and thus parents from many areas concentrate in one specific building. When you re-distribute these births out to their respective homes around the county, the zip code becomes much more important.

I used my mailing address zipcode, without even realizing it wasn't my birth zipcode :)

Where are you searching for your zip code and birth date? I couldn't find anything on the Data Privacy Lab site. :\

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#64

Post by tlb »

zeratul wrote: post #48

De-identified patient data does NOT have DOB or ZIP codes: http://www.hipaa.com/2009/09/hipaa-protected-health-informat... Sex, gender or age is allowed.

But if you poll the database every day to see when age increases, you can get DOB.

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#65

Post by nkassis »

mbrubeck wrote: post #11
Zakuzaa wrote: post #8

I don't find that number surprising. DOB along with zip is pretty good data. What I'd find amusing would be how much of the population is uniquely identifiable by browser+plugins+os+resolution.

The EFF has done some research on that question: https://panopticlick.eff.org/

hmmmm..... Looks like I should use IE instead of chrome.

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#66

Post by nkassis »

tlb wrote: post #64
zeratul wrote: post #48

De-identified patient data does NOT have DOB or ZIP codes: http://www.hipaa.com/2009/09/hipaa-protected-health-informat... Sex, gender or age is allowed.

But if you poll the database every day to see when age increases, you can get DOB.

Well... if the records are released I don't think they should change. Someone record at 50 isn't the same as at 60. Unless these are live datasets.

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#67

Post by _delirium »

This is sort of the reverse of the "curse of dimensionality", for those interested in machine learning (http://en.wikipedia.org/wiki/Curse_of_dimensionality). From an ML perspective, as you add dimensions to a dataset, the amount of data you'd need to accurately model the data without overfitting (i.e. without memorizing specific details of the sample rather than underlying trends) grows very fast, because due to the combinatorics you end up with extremely sparse coverage of the overall possibility space even with huge data sets.

The reverse of that phenomenon is that, given a data set in a high-dimensional space (even 3 dimensions, if each dimension has more than a few bits of entropy), it will cover the dimensions very sparsely (even if it's large!), and therefore it's relatively easy to recover specific details of the sample from the aggregate statistics.

edit: Well, I was hoping this might be a new insight, but in fact there's a good 2005 paper exploring that connection in much more detail: http://www.vldb2005.org/program/paper/fri/p901-aggarwal.pdf

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#68

Post by delluminatus »

mbrubeck wrote: post #11
Zakuzaa wrote: post #8

I don't find that number surprising. DOB along with zip is pretty good data. What I'd find amusing would be how much of the population is uniquely identifiable by browser+plugins+os+resolution.

The EFF has done some research on that question: https://panopticlick.eff.org/

Interesting -- using NoScript reduced my "bits of information" from 20 to 14. I would have expected it to increase the uniqueness of my browser.

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#69

Post by caf »

tokenadult wrote: post #56
Zimahl wrote: post #18

Earlier quoted context omitted.

October 8th is an odd date. Since it's my birthday as well (long before 1990 though) I have taken note that I run across very few with that birthday, let alone early October birthdays. Is it statistically a low-birth month? I'd love to know. On a side note, Happy 21st Birthday in just over a month!

October 8th is actually in the top 100 of birthdates by frequency of births, according to the best data table I have seen on the subject. http://www.nytimes.com/2006/12/19/business/20leonhardt-table... Of course February 29th is dead last in frequency. Most early October birthdays are above the median in frequency.

Note the group of high rankings from 8-Sep, tailing off in early October - all those NYE babies ;)

Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}

#70

Post by jcdreads »

russell wrote: post #44
jdp23 wrote: post #5

Here's some backstory, from the 1990s when the Massachusetts' Group Insurance Commission released "anonymized" health data: ''At the time GIC released the data, William Weld, then Governor of Massachusetts, assured the public that GIC had protected patient privacy by deleting identifiers. In response, then-graduate student Sweeney started hunting for the Governor’s hospital records in the GIC data. She knew that Gove…

I had to deal with the converse of the problem when I was working for a company that ran the vehicle smog checking for the state of Illinois. A driver's license id was composed of birthdate, sex, and some location information (I dont remember exactly what) for the reasons mentioned by jdp23 it worked quite well, except for identical twins, who had the same driver's license id. (Registrations ofter were processed by h…

My wife had her wallet stolen on the T (in Boston) once, which would have been manageable but for the fact that Massachusetts driver's license ID's were your Social Security number. The thief (or someone) then used her SSN to open a series of credit cards and rack up a bunch of charges—computers, car repairs, etc.—the aftermath of which took at least a few hundred hours of hassle spread over three or four years to clear up. (Getting our first mortgage somewhere in there made it even more painful.) Mercifully, it eventually cleared up.

MA driver's licenses no longer use SSN as far as we can tell.

Post reply on HN