Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noise infusion banned from statistical products published by Census Bureau

desfontain.es

Post reply on HN
Print view View original

11–20 of 662 posts

Re: Noise infusion banned from statistical products published by Census Bureau

#11

Post by toast0 »

Pragmata wrote: post #3

Frankly i see no reason to keep this data private. They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc... Fundamentally this is public data. If it's to dangerous to make public, it's too dangerous to collect, and people should be aware of exactly what it is. There are very few things that the state has data on that should not be made public. Census data is s…

> They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc...

They do. After a substantial delay. Pretty handy for geneological research, while protecting privacy for the living.

Re: Noise infusion banned from statistical products published by Census Bureau

#12

Post by tbrownaw »

> Differential privacy makes this trade-off explicit, and thus impossible to ignore. Maybe banning it is a way of pretending that the problem doesn't exist, in the hope that it will go away?

Or it's saying that one of these conflicting goals is more valuable than the other, and so shouldn't be sacrificed for it.

Re: Noise infusion banned from statistical products published by Census Bureau

#13

Post by asolove »

The replies here arguing we should publish it all are wild in the worst kind of first-order thinking way.

It’s a census: it just asks questions.

If you start publishing and weaponizing the data against people with various attributes, they’ll just lie or not answer. And then you are left with worse than nothing: bad data people try to act on.

Re: Noise infusion banned from statistical products published by Census Bureau

#14

Post by abletonlive »

simonw wrote: post #9
abletonlive wrote: post #4

There will be a bunch of people that start off with the premise that this data should be private and make following arguments based on this premise. So I'll just go ahead and ask, give me good reasons why this data should be private? My guess is that most of you think we should be counting illegals because they should have representation. And I reject that

How about we should be "counting illegals" so that we know how many of them there are? (Do you reject that? As someone who uses the phrase "counting illegals" I imagine you would be interested in knowing what that number is.)

First off the census is used for determining how many seats are used for congressional apportionment and allocating federal funds.

So unless you're willing to also say that counted illegals cannot used for either of those, then you're just being obtuse.

But if we can agree that they cannot be used for that then sure, lets identify and count them. If we can't identify (make non-private) and count them then why should we trust that those counts are accurate?

Re: Noise infusion banned from statistical products published by Census Bureau

#15

Post by mobeets »

simonw wrote: post #6
Pragmata wrote: post #3

Frankly i see no reason to keep this data private. They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc... Fundamentally this is public data. If it's to dangerous to make public, it's too dangerous to collect, and people should be aware of exactly what it is. There are very few things that the state has data on that should not be made public. Census data is s…

How hard have you thought about this? The biggest challenge with running a census is getting people to trust you enough to answer your questions. A lot of census questions are sensitive. The ACS covers topics like citizenship status, disabilities, income, SNAP assistance, languages spoken at home. If you want accurate information about the people who live in your country you need the census process to feel as safe fo…

Thank you for writing a much more thoughtful reply to this comment than I was drafting

Re: Noise infusion banned from statistical products published by Census Bureau

#16

Post by jmole »

Ban it from the dataset, add it to the analysis. You can choose your own flavor of noise.

I don't know what the political undertones are here, but at some level you need to have actual ground truth, including "this person/household declined".

Publishing raw data though? That seems like shooting yourself in the foot from a national security perspective, not to mention all the other reasons not to do it.

Re: Noise infusion banned from statistical products published by Census Bureau

#17

Post by Cyph0n »

abletonlive wrote: post #4

There will be a bunch of people that start off with the premise that this data should be private and make following arguments based on this premise. So I'll just go ahead and ask, give me good reasons why this data should be private? My guess is that most of you think we should be counting illegals because they should have representation. And I reject that

It’s because people are significantly more likely to lie or omit some facts if you don’t guarantee their privacy, which means your census data ends up being worth less than a pile of shit.

The alternative is to water down the census questions, which also leads you down the same path (i.e. manure as data).

Re: Noise infusion banned from statistical products published by Census Bureau

#18

Post by jonhohle »

simonw wrote: post #6
Pragmata wrote: post #3

Frankly i see no reason to keep this data private. They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc... Fundamentally this is public data. If it's to dangerous to make public, it's too dangerous to collect, and people should be aware of exactly what it is. There are very few things that the state has data on that should not be made public. Census data is s…

How hard have you thought about this? The biggest challenge with running a census is getting people to trust you enough to answer your questions. A lot of census questions are sensitive. The ACS covers topics like citizenship status, disabilities, income, SNAP assistance, languages spoken at home. If you want accurate information about the people who live in your country you need the census process to feel as safe fo…

This seems’s like an issue created by congress. the constitution only requires a headcount by state. Maybe they should use another mechanism to collect demographic data. Since the concern is not about representation, but allocation, tax returns seem like an obvious alternative and they are already private and collected at a much more granular level.

Re: Noise infusion banned from statistical products published by Census Bureau

#19

Post by ghaff »

delichon wrote: post #5

The dueling political demands of accuracy and privacy are simply incompatible at some level. After reading this, maybe Hanlon's Razor isn't the right standard. Besides malice and stupidity, there is impossibility. Some problems just aren't solvable under certain constraints. I don't envy the statisticians tasked with finding a politically palatable solution to a math problem.

There's a ton of information in the US that is accessible to various degrees--especially through the the deep web much less background investigations. Unless you're a wealthy person who can set up various levels of trusts you can't really hide them.

You can of course disagree about what what should actually be part of a transparent public record. (Though I suspect a lot of people post-date what was generally available in a "phone book.")

Re: Noise infusion banned from statistical products published by Census Bureau

#20

Post by CAP_NET_ADMIN »

Pragmata wrote: post #3

Frankly i see no reason to keep this data private. They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc... Fundamentally this is public data. If it's to dangerous to make public, it's too dangerous to collect, and people should be aware of exactly what it is. There are very few things that the state has data on that should not be made public. Census data is s…

1. People give the information to the government under the expectation that this data is to be kept private or used in such a way that individual targeting is made impossible, you break that expectation and people will lie or won't give you this data.

2. Without noise injection it's rather simple to do statistical attacks to reverse engineer individual entities.

3. This data is and has already been used in the past to undermine democratic systems by targeting and disenfranchising minorities, as well as gerrymandering the US to hell.

4. "Too dangerous to make public, too dangerous to collect" - this is a false dichotomy. To govern effectively you need sensitive data, but it should be collected and used in a way that's safe for the individuals.

5. Macro level aggregates don't need individual exposure, that's why noise, anonymization and statistical functions are fine.

Post reply on HN