Debunking the Myth of "Anonymous" Data • Hacker News Board

Re: Debunking the Myth of "Anonymous" Data

#41

Post by e12e » Sat, Nov 11, 2023, 10:15 PM UTC

jeffbee wrote: post #39

willseth wrote: post #21

Earlier quoted context omitted.

Another fun one was the Netflix Prize, where Netflix published an anonymized dataset, but some researchers at UT Austin were able to de-identify many/most of the users in the dataset by linking them to IMDB profiles based on preferences.

There's no way most of the users in the Netflix data had IMDb accounts.

From a skim, it looks like their main point is that with 8 movie ratings, it's trivial to link two datasets (eg anonymized Netflix rating to public IMDb profile).

https://scholar.google.com/citations?view_op=view_citation&h...

https://systems.cs.columbia.edu/private-systems-class/papers...

Re: Debunking the Myth of "Anonymous" Data

#42

Post by candiddevmike » Sat, Nov 11, 2023, 10:22 PM UTC

__MatrixMan__ wrote: post #36

willseth wrote: post #16

I work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using comb…

I'm usually the first to defend the EFF but I agree that they've gone a bit far here. The anonymization script which I wrote for my company just replaces every string in the customer's database with a cryptographic hash--except a list of strings like "failed" and "success". So unless your city is named "success", it's going to be missing from the dataset. It's a bit bewildering to actually run the app in this mode, b…

These scripts always have the best intentions but typically end up neglected and suddenly production data is on a dev laptop (at least, in my experience, both as an employee and a customer).

Make mock data inputs or snapshot data based on internal/QA usage, please!

Re: Debunking the Myth of "Anonymous" Data

#43

Post by ralferoo » Sat, Nov 11, 2023, 10:30 PM UTC

__MatrixMan__ wrote: post #36

willseth wrote: post #16

I work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using comb…

I'm usually the first to defend the EFF but I agree that they've gone a bit far here. The anonymization script which I wrote for my company just replaces every string in the customer's database with a cryptographic hash--except a list of strings like "failed" and "success". So unless your city is named "success", it's going to be missing from the dataset. It's a bit bewildering to actually run the app in this mode, b…

> replaces every string in the customer's database with a cryptographic hash > So unless your city is named "success", it's going to be missing from the dataset.

You've made the typical mistake of thinking that just because you've made it harder, you've also made it anonymous.

For instance, you might have replaced "Chicago" with 9cfa1e69f507d007a516eb3e9f5074e2, but if for instance a lot of people with that tokenised city name also have transactions for a store that only exists in Chicago, you can infer with a reasonable degree of accuracy the reverse mapping. A couple such data points, and you can be almost certain.

If you know when you made a couple of transactions that are in the database, and you can find a set of likely options for each time, and then see what fields they have in common. Once you know your data, you can infer a whole load of the reverse mappings for various fields.

If those transactions involve another user, you can start to correlate those mappings with data you know about them, and start to build up a web of transactions that person had even if you don't necessarily know yet who the other users they interact with yet.

All this is possible without the use of rainbow tables, but chances are your hash function is a standard one, so with a single known mapping you can work out which hash function you chose, and look for speculative entries in the data. e.g. let's just look for MD5("Chicago"), SHA1("Chicago"), SHA256("Chicago") and see if there are any matches. If there are, we can use that hash function and trivially create a rainbow table for every city in the US, first names, surnames, etc.

Re: Debunking the Myth of "Anonymous" Data

#44

Post by chaps » Sat, Nov 11, 2023, 10:41 PM UTC

willseth wrote: post #16

I work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using comb…

  That is usually referred to as "pseudonymization"

For your world, maybe. For most things with a public face, "anonymization" is what's used, not "pseudonymization". So yes, this article is referring to its more-common use.

Re: Debunking the Myth of "Anonymous" Data

#45

Post by Xelynega » Sat, Nov 11, 2023, 10:52 PM UTC

willseth wrote: post #28

genewitch wrote: post #23

Earlier quoted context omitted.

In the interest of charitable-ness, can you define "noise" and explain how rudimentary denoising algorithms that have existed since the 1960s can't penetrate it? Time and time again people think they've anonymized data and they're always proven incorrect. Just come to grips with the fact you're participating in the sale of my (or whoever's) private data. Hope you sleep well!

"Denoising" algorithms can't remove this kind of noise. It doesn't work like that. You're simply mistaken. Generally the state of the art for adding noise to data is differential privacy or microaggregation. In the case of differential privacy it's typically gaussian or laplacian noise, but it is not a trivial application. Noise applied through microaggregation is not a mathematical function because microaggregation…

Who do you think "data brokers and ad targets" purchase data from?

I don't understand how you could help companies protect the private data they collect from users and think that you're not facilitating the collection and sale of it.

Re: Debunking the Myth of "Anonymous" Data

#46

Post by dfcowell » Sat, Nov 11, 2023, 10:57 PM UTC

jandrewrogers wrote: post #29

jruohonen wrote: post #2

A good popular take, but they, either intentionally or out of ignorance, omit newer, proven techniques like differential privacy.

Techniques like differential privacy do not work for some types of data models, including many of the more interesting/risky ones. I've never seen a technique that can deliver an analytical model at scale that is both analytically effective and anonymous while also robust against sophisticated de-anonymization attacks. There is no good theoretical foundation to suggest that such things are possible. Most modern techn…

If your scale is large enough and you don’t care about identifying individuals, synthetic data does this fairly effectively.

Re: Debunking the Myth of "Anonymous" Data

#47

Post by Taek » Sat, Nov 11, 2023, 11:01 PM UTC

willseth wrote: post #16

I work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using comb…

> *There is always a tradeoff between privacy and utility. The only way to achieve 100% private data is 100% noise, but the privacy-utility tradeoff curve isn't linear, and you can still achieve very good utility and very good privacy in many cases, especially with the best tools. Methods are also improving over time, reducing the impact of the tradeoff.

This is the crux of the problem. In every situation where I've seen it deployed, "anonymization" is as far up the utility curve as the corporate interest can get while still credibly calling their data anonymized. The standards and rules that are in place for restricting what types of data can be called "anonymized" to the public are so far up the utility curve that there's functionally no privacy at all.

I don't know what your startup does or what steps it takes to make that tradeoff responsibly, but assuming that you are responsible, you are one of the few, and most of the customers in the world would rather use a client that is using the word "anonymous" more irresponsibly because:

1. they can get away with it

2. there's a lot more business value in obfuscating data as little as possible

The EFF article is bang-on here, because the majority of the "anonymization" in the space isn't being done by responsible folks (like yourself, presumably), it's being done by psychopathic business interests.

Re: Debunking the Myth of "Anonymous" Data

#48

Post by ImPostingOnHN » Sat, Nov 11, 2023, 11:04 PM UTC

Xelynega wrote: post #45

willseth wrote: post #28

Earlier quoted context omitted.

"Denoising" algorithms can't remove this kind of noise. It doesn't work like that. You're simply mistaken. Generally the state of the art for adding noise to data is differential privacy or microaggregation. In the case of differential privacy it's typically gaussian or laplacian noise, but it is not a trivial application. Noise applied through microaggregation is not a mathematical function because microaggregation…

Who do you think "data brokers and ad targets" purchase data from? I don't understand how you could help companies protect the private data they collect from users and think that you're not facilitating the collection and sale of it.

From sources that don't use differential privacy to mask the relevant features.

Can you maybe share your mathematical opposition to differential privacy? It's provably effectively.

Re: Debunking the Myth of "Anonymous" Data

#49

Post by uoaei » Sat, Nov 11, 2023, 11:18 PM UTC

__MatrixMan__ wrote: post #36

willseth wrote: post #16

I work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using comb…

I'm usually the first to defend the EFF but I agree that they've gone a bit far here. The anonymization script which I wrote for my company just replaces every string in the customer's database with a cryptographic hash--except a list of strings like "failed" and "success". So unless your city is named "success", it's going to be missing from the dataset. It's a bit bewildering to actually run the app in this mode, b…

> So unless your city is named "success", it's going to be missing from the dataset.

https://geotargit.com/called.php?qcity=Success

> There are 11 places in the world named Success!

Negligence of all possible corner cases is common in handcrafted, "well-intentioned" approaches such as this. I put well-intentioned in quotes because assuming you know all the issues and premature confidence in the solutions demonstrates a lack of intention to actually get it right. Hubris permeates the mindscape of move-fast-break-things types.

Sometimes the failures are more obvious such as those pointed out by sibling comments. Part of the EFF's general point is it's a complicated business with so many pitfalls that it's a shame we have to learn by trial and error when people's private data is on the line. There's a reason many large companies have entire departments exclusively focused on privacy in their data stores.

Re: Debunking the Myth of "Anonymous" Data

#50

Post by willseth » Sat, Nov 11, 2023, 11:20 PM UTC

Xelynega wrote: post #45

willseth wrote: post #28

Earlier quoted context omitted.

"Denoising" algorithms can't remove this kind of noise. It doesn't work like that. You're simply mistaken. Generally the state of the art for adding noise to data is differential privacy or microaggregation. In the case of differential privacy it's typically gaussian or laplacian noise, but it is not a trivial application. Noise applied through microaggregation is not a mathematical function because microaggregation…

Who do you think "data brokers and ad targets" purchase data from? I don't understand how you could help companies protect the private data they collect from users and think that you're not facilitating the collection and sale of it.

Almost literally every company with human customers collects user data because it’s necessary to run a business. Very few of those have a business model where the purpose is to profit from the sale of that data. Every one of our customers is treating first party data for first party use cases, which is far more common than buying or selling data. How many companies have prod user data and developers that need an up to date prod-like database? A ton. Many just copy prod and distribute to devs. The good ones try to keep scripts for populating the database with test data up to date, but that’s a nightmare. Anonymization is an elegant solution to this problem. There are countless other use cases where organizations need to use their customer data internally and (should) want to protect identities.

Most companies don’t want to sell their user data because they consider it competitive, and most data brokers don’t want to anonymize because why would they if they’re already selling raw data? They aren’t interested in our software because we prevent singling out of identities.