If we want more anonymized datasets, maybe we should be asking for approx. age or year of birth instead. I doubt that most things that ask for DOB need anything more than granular than that.
87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
31–40 of 102 posts
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#32Here's some backstory, from the 1990s when the Massachusetts' Group Insurance Commission released "anonymized" health data: ''At the time GIC released the data, William Weld, then Governor of Massachusetts, assured the public that GIC had protected patient privacy by deleting identifiers. In response, then-graduate student Sweeney started hunting for the Governor’s hospital records in the GIC data. She knew that Gove…
If only 6 people out of 54000 shared Weld's birthday, she was just lucky (not that this invalidates the main point).
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#33While I find this fascinating, I'm sure this is old news to people in the business, like the people that run the grocery/chain store savings programs. What I'd be really interested to see is why this works, and what it tells us about distribution of population by zip code. I'd imagine the places where this doesn't work as well, are the most densely populated zip codes, where the likely hood of duplicates on the given…
Here's a quick back of the envelope calculation:
The US population is 300 million and there's about 43,000 zip codes in the US. Assuming an even distribution, that's about 7,000 people per zip code. Cut it down to just one gender and we're at 3,500. The probability of a chosen person in that group having a a unique birthday is (364/365)^3500 = 0.0068%. Now, if we say that the person could have been born in one of the past sixty years, we get (21914/21915)^3500 = 85% as the probability that this person has been uniquely identified.
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#34If we want more anonymized datasets, maybe we should be asking for approx. age or year of birth instead. I doubt that most things that ask for DOB need anything more than granular than that.
I wish. Unfortunately, most sites need this for legal reasons, to determine whether or not the user is above or below a certain age.
One could theoretically ask "how old are you" but then you don't know when to lift limitations. Say a user joins six months before their 18th birthday. Are you going to block 18+ content for a whole year from then?
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#35There were no other males, born on october 8th 1990, in Lakeview, Chicago. I'm a little flabbergasted, to be frank.
October 8th is an odd date. Since it's my birthday as well (long before 1990 though) I have taken note that I run across very few with that birthday, let alone early October birthdays. Is it statistically a low-birth month? I'd love to know. On a side note, Happy 21st Birthday in just over a month!
As another October 8th birthdate with the opposite impression of how many October kids there are, I had to look this up.
8.62% of all live births in the US during 2006 occurred during October [1]. No single month deviated significantly from it's expected percentage.
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#36I'd love to see someone with a large dataset containing those variables to see what % of their set overlaps on those.
About a year ago, I did a project for de-duping customer entered contact info. We had less than 100k records for any particular region (Bay Area, New York, etc.) From what I recall, we were able to confirm identity in about 80% of cases using Name, DOB, and zip code.
By far the best metric is phone number, that bumped us up over 97%. If the comparison was limited to only three metrics, then we would use Name, DOB, and phone number. In reality, we also compare zip code, street address, and contacts (like friends on facebook), in that order.
It would be interesting to see if on average, people change their phone number less often than their name (marriage) or address (renters).
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#37Here's some backstory, from the 1990s when the Massachusetts' Group Insurance Commission released "anonymized" health data: ''At the time GIC released the data, William Weld, then Governor of Massachusetts, assured the public that GIC had protected patient privacy by deleting identifiers. In response, then-graduate student Sweeney started hunting for the Governor’s hospital records in the GIC data. She knew that Gove…
If only 6 people out of 54000 shared Weld's birthday, she was just lucky (not that this invalidates the main point).
Birthdate includes year
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#38Here's some backstory, from the 1990s when the Massachusetts' Group Insurance Commission released "anonymized" health data: ''At the time GIC released the data, William Weld, then Governor of Massachusetts, assured the public that GIC had protected patient privacy by deleting identifiers. In response, then-graduate student Sweeney started hunting for the Governor’s hospital records in the GIC data. She knew that Gove…
If only 6 people out of 54000 shared Weld's birthday, she was just lucky (not that this invalidates the main point).
Including birth year helps significantly. Assuming an even distribution of births, you'd expect 54000/365 ~ 148/day. Then knowing gender lets you cut that in half. Dividing again by the age distribution of voters gets that down to a reasonable number.
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#39There were no other males, born on october 8th 1990, in Lakeview, Chicago. I'm a little flabbergasted, to be frank.
October 8th is an odd date. Since it's my birthday as well (long before 1990 though) I have taken note that I run across very few with that birthday, let alone early October birthdays. Is it statistically a low-birth month? I'd love to know. On a side note, Happy 21st Birthday in just over a month!
Interesting probabilities on x random people sharing the same birthday:
Re: 87% of the U.S. Population are uniquely identified by {DOB, gender, zip}
#40Earlier quoted context omitted.
Birth location =/= mailing address zipcode. Babies are usually born in hospitals, and thus parents from many areas concentrate in one specific building. When you re-distribute these births out to their respective homes around the county, the zip code becomes much more important.
In towns like mine, however, 40,000 people share the same zip code with the hospital. I'm sure there are quite a few more town like mine throughout the US.
[deleted]