Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simulating identification by zip code, sex, and birthdate

johndcook.com

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Simulating identification by zip code, sex, and birthdate

#31

Post by nkurz »

pinneycolton wrote: post #26
nkurz wrote: post #16

While this is intended as just a proof of concept, I think this technique is mathematically flawed. You can't just use the average number of people in each zipcode, because if there is great variation in the number of people per zipcode, a randomly sampled person is more likely to be in a "large" zipcode than a small one. Consider the case where we have 100000 people, and 10 zipcodes, each with an average of 10000 pe…

I work with this type of data and I assure you that the results are quite plausible. The original hypothesis was tested against US census data. See "Experiment B" here: https://dataprivacylab.org/projects/identifiability/paper1.p... I'll add that there are far fewer live births, per day, in the US than there are zip codes. I agree that some highly populated areas that are problematic, but this may be the only reason…

It may have gotten lost in the revisions of my comment, but I was consciously not trying to argue that Sweeney's numbers were wrong, only that Cook's explanation was lacking since it doesn't discuss distribution. I hadn't yet looked at the paper.

That said, looking at the paper you linked now, I don't see how Cook's simulation (simulation, not explanation) and Sweeney's paper can both be correct. Cook got 84-85% identifiable assuming uniform age distribution and identical population per zipcode. At the bottom of Figure 14, Sweeney says 87% for the US as a whole.

Shouldn't any non-uniformity (of zip code population or age clustering) act only to reduce the percent of the population that is identifiable? That is, shouldn't Cook's simulation with flat age distribution and equal zipcode populations be an upper bound on identifiability? Since Cook's simulation code looks fine, this makes me suspect that there's something off about Sweeney's analysis.

Is the 87% perhaps an average of the state percentages, and not properly weighted by state population? Or maybe an average across age classes not weighted by population of that class? Oh, I don't know about those, but maybe I see a bigger issue now...

In Section 4.3.1, Sweeney defines the "Number of subjects uniquely identified in a subdivision of a geographical area". But this isn't a simulation like Cook did, she's just using a binary yes/no depending on whether the subpopulation in each age class exceeds a numerical threshold:

  if population(zi, a) ≥ |Qa|, then ID_aZi= population(zi, a)   
  else ID_aZi = 0.
While it's nice that it's clearly defined, I don't think this yields a "percent identifiable" that matches up with Cook's simulation, nor with any common usage of the term. Also (while I'm being picky) isn't the definition backward? If we were to go with this arbitrary definition, wouldn't we want Id_zi to equal zero when the population is less than the threshold? I presume the direction of equality is just a typo, but if the paper is using a hard threshold rather than some more rigorous approach like Cook's simulation, this seems like a major flaw in interpretability of the results.

Re: Simulating identification by zip code, sex, and birthdate

#32

Post by stochastic_monk »

dlhavema wrote: post #24
cmroanirgo wrote: post #21

Earlier quoted context omitted.

In some cultures, sex is what you do, gender is what you have. So for me, when asked for sex, my answer is "yes, please". When asked for gender it's M/F. I understand that the US is different in this regard.

I agree sex is a verb and also an attribute...

Sex is only a verb in the case of determining the sex of an animal.

Re: Simulating identification by zip code, sex, and birthdate

#33

Post by nkurz »

nkurz wrote: post #31
pinneycolton wrote: post #26

Earlier quoted context omitted.

I work with this type of data and I assure you that the results are quite plausible. The original hypothesis was tested against US census data. See "Experiment B" here: https://dataprivacylab.org/projects/identifiability/paper1.p... I'll add that there are far fewer live births, per day, in the US than there are zip codes. I agree that some highly populated areas that are problematic, but this may be the only reason…

It may have gotten lost in the revisions of my comment, but I was consciously not trying to argue that Sweeney's numbers were wrong, only that Cook's explanation was lacking since it doesn't discuss distribution. I hadn't yet looked at the paper. That said, looking at the paper you linked now, I don't see how Cook's simulation (simulation, not explanation) and Sweeney's paper can both be correct. Cook got 84-85% iden…

Now that edit window is over, I finally noticed the massive error in my wording in the second to last sentence. Instead, let's pretend I wrote "why would we want Id_zi to equal zero when the population is less than the threshold?" It would also be good to note again that I haven't read the paper closely, and very well might be misinterpreting what it is doing.

---

But since I'm still in this edit window, I'll add an update here. I downloaded the per zip population data from here: https://blog.splitwise.com/2013/09/18/the-2010-us-census-pop.... Then I wrote a quick Perl program (parallel to Cook's Python simulation) but using the actual per zipcode populations rather than a fixed average. After confirming Cook's 84% number with the fixed population, I ran it on the actual populations (but still with a flat distribution for age and sex) and got 63% uniques.

Presumably this number would drop somewhat further with actual age distributions, but I don't know how far exactly. My current belief is that Sweeney's paper does a good job of calling attention to the fact that risk of identification is high, but the methodology and exact numbers should not be trusted. The actual percentage of Americans identifiable by (zip, dob, sex) is large, but something less than 63%. It might be interesting to run the simulation with actual age bracket data, but I didn't find this in any easy to download format, so I think I'll stop here.

Re: Simulating identification by zip code, sex, and birthdate

#34

Post by cmroanirgo »

bloak wrote: post #27
cmroanirgo wrote: post #21

Earlier quoted context omitted.

In some cultures, sex is what you do, gender is what you have. So for me, when asked for sex, my answer is "yes, please". When asked for gender it's M/F. I understand that the US is different in this regard.

It's generally understood that sex is biological and gender is social, or grammatical: it makes sense for those two to go together as in many languages, including standard English, you need to know a person's gender in order to talk about them easily. In the UK, at least, official documents specify "sex", not "gender". We may be slowly moving towards a world in which sex is officially recorded only in medical records…

Wow. Cultural differences get me downvoted.

All the forms in my country, especially the government ones, use 'gender' and not 'sex'. For me, when I see 'sex' on a form, it's jarring to say the least.

I'm Australian.

Re: Simulating identification by zip code, sex, and birthdate

#35

Post by kolinko »

Fun fact, in Poland (population ~35M), every person has a public identifier in a format of:

YYMMDDXXXXC

Where YYMMDD is date of birth, XXXX a number, and C a checksum.

https://en.wikipedia.org/wiki/PESEL#Format

It works very well, and - since it's for identification, not authentication - you don't have to worry about it leaking out too much, like you would with SSN.

Re: Simulating identification by zip code, sex, and birthdate

#36

Post by seandougall »

dlhavema wrote: post #23
jMyles wrote: post #4

Earlier quoted context omitted.

I think that your alteration of the title is good, but why did you change "sex" to "gender"?! The study appears to be about sex as a matter of public record, so you've added inaccuracy, or at least ambiguity, to the title.

Is sex male/female and gender "what I identify as"? I've always thought of them in this context as meaning the same thing...

Biological sex is most often male or female, but not always. People can be born intersex (with genitalia that are neither clearly male nor female, which is often “corrected” surgically at birth, and/or with genitalia that don’t match what’s expected from their chromosomes, and/or with chromosome combinations other than XX or XY). Chimerism can result in different parts of the body having different sex chromosomes. It’s not strictly monolithic or binary.

That aside, yes, “sex” broadly refers to biology in some way, though often oversimplified, while “gender” refers to identity and expression (which also don’t necessarily go hand in hand).

Re: Simulating identification by zip code, sex, and birthdate

#37

Post by nv-vn »

kolinko wrote: post #35

Fun fact, in Poland (population ~35M), every person has a public identifier in a format of: YYMMDDXXXXC Where YYMMDD is date of birth, XXXX a number, and C a checksum. https://en.wikipedia.org/wiki/PESEL#Format It works very well, and - since it's for identification, not authentication - you don't have to worry about it leaking out too much, like you would with SSN.

What is the typical use case of these numbers?

Re: Simulating identification by zip code, sex, and birthdate

#38

Post by bloak »

cmroanirgo wrote: post #34
bloak wrote: post #27

Earlier quoted context omitted.

It's generally understood that sex is biological and gender is social, or grammatical: it makes sense for those two to go together as in many languages, including standard English, you need to know a person's gender in order to talk about them easily. In the UK, at least, official documents specify "sex", not "gender". We may be slowly moving towards a world in which sex is officially recorded only in medical records…

Wow. Cultural differences get me downvoted. All the forms in my country, especially the government ones, use 'gender' and not 'sex'. For me, when I see 'sex' on a form, it's jarring to say the least. I'm Australian.

That's interesting. I wonder what's going on there. Do the words "sex" and "gender" have slightly different meanings for an Australian, or are the Australian forms asking a slightly different question from the UK ones?

But your passport says "Sex / Sexe" just below your date of birth, right?

The design of passports is rather precisely defined by an international standard. Interestingly, and perhaps relevantly, Australia is one of a dozen countries that routinely allows its citizens to have an 'X' instead of an 'F' or 'M' in that field. There is an ongoing legal case to obtain that right in the UK (Christie Elan-Cane).

Does your driving licence indicate "sex" or "gender"? A UK driving licence shows neither, and it doesn't give a "title", either: just full name, address, date of birth, and photo. In most cases that would allow one to guess the sex and gender of the holder, of course, but not always.

Re: Simulating identification by zip code, sex, and birthdate

#39

Post by garaetjjte »

kolinko wrote: post #35

Fun fact, in Poland (population ~35M), every person has a public identifier in a format of: YYMMDDXXXXC Where YYMMDD is date of birth, XXXX a number, and C a checksum. https://en.wikipedia.org/wiki/PESEL#Format It works very well, and - since it's for identification, not authentication - you don't have to worry about it leaking out too much, like you would with SSN.

Sadly, it is used for authentication sometimes. Name, PESEL and mother's maiden name is sometimes enough to authenticate on bank helpline.
Post reply on HN