Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The False Allure of Hashing for Anonymization

gravitational.com

Post reply on HN
Print view View original

91–100 of 106 posts

Re: The False Allure of Hashing for Anonymization

#91

Post by aidos »

pcarolan wrote: post #88
aidos wrote: post #62

I saw a case a few years ago where the management of a company I knew were worried that the sales team were covering their mistakes and lying about it to blame the (external) dev team's code. They asked me to take a look into it one morning. At first glance there didn't seem to be a lot to go on. There was no auditing in the application itself so I focused on the nginx logs. It's amazing how clear of a picture you ca…

I think the bigger problem was the culture that company had in place that would lead people to do that.

Absolutely! Then again, no more broken than a number of other places I’ve worked over the years. I could write a book on that subject :-)

Re: The False Allure of Hashing for Anonymization

#92

Post by AndrewSChapman »

If our goal is true anonymisation, that is, even the host cannot know who the data belongs to, why are we hashing data at all, and not completely removing it? Replace the pii (name, email address, phone etc) with a fixed number of *'s. There's no reversing or guessing that.

If we are wanting information to be readable by some people in some circumstances, that's not anonymisation: that's data protection and an entirely different problem.

Re: The False Allure of Hashing for Anonymization

#93

Post by javajosh »

When addressing the solution of adding data (salt) I find the authors counter-argument unconvincing:

   Don’t get me wrong, this does make it significantly harder 
   to attack a leaked database to unmask every user, but the 
   resources required to do so or target specific users are 
   within the reach of many adversaries.
I don't see how it's more feasible to reverse hash(known_user+salt) than it is to dereference hash(salt), and even state level actors can't do anything but attempt to brute-force hash(salt). IOW without more behind the author's assertion, I don't buy it that adding more data to the data you want to protect is insufficient protection, even against known targets.

Re: The False Allure of Hashing for Anonymization

#94

Post by beagle3 »

pcarolan wrote: post #88
aidos wrote: post #62

I saw a case a few years ago where the management of a company I knew were worried that the sales team were covering their mistakes and lying about it to blame the (external) dev team's code. They asked me to take a look into it one morning. At first glance there didn't seem to be a lot to go on. There was no auditing in the application itself so I focused on the nginx logs. It's amazing how clear of a picture you ca…

I think the bigger problem was the culture that company had in place that would lead people to do that.

Sometimes it's just what the people bring with them, even if the company has "good" culture (whatever that means).

And even in companies with the best culture, I would expect such things to happen if the cost of a mistake is comparable to a person's yearly salary or above that.

Re: The False Allure of Hashing for Anonymization

#95

Post by lbriner »

The link to Cryptographic Right Answers is really helpful and the kind of article that it would be nice to make the general "go-to" for those of us who know enough but not enough to do it ourselves!

What I didn't like was the continual reference to AWS as if it is the only provider available, without qualifying whether it is specifically an AWS product that solves the problem or whether it is an example of using a cloud service to transfer the risk. There are many alternatives to AWS load balancers and Key Management systems, so the advice is tainted sigh

Re: The False Allure of Hashing for Anonymization

#96

Post by e_proxus »

benmowa wrote: post #52
jacquesm wrote: post #22

The idea that data is a corporate asset has to die. Data is a corporate liability.

i agree that data is a liability, however even my plumber's truck can kill someone and can be considered a liability. regardless, the truck is not something he can do business without. I agree that companies should fear the data they retain much more than they do today.

To be fair, companies gather a lot more data than they need to to do business these days.

Re: The False Allure of Hashing for Anonymization

#97

Post by PeterisP »

nebulous1 wrote: post #85
PeterisP wrote: post #36

SHA256 pretty much ensures that you have a unique hash for every value - and that's a feature you don't want for anonymization. So why not simply take the first few bytes of a SHA256, a small enough set to ensure that collisions not only might happen but will happen? I mean, that's a required feature to ensure anonymization, not just pseudonymization - if you can select a whole trail of events for ID #123 and be sure…

I'm not sure you could make the data statistically meaningful and have too many false positives to deanonymize an id. I think you're basically suggesting randomly grouping the ids so they average X real ids per grouped ID. At least if you just did it randomly instead of by hashing then there would be no danger of a dictionary attack.

The expectation is that a brute force attack would try orders of magnitude more IDs than you actually have. It means that if a random ID is 90% likely to have a unique hash and 10% likely to map to one of your real IDs, then your real data won't have that many collisions, however, if someone does a brute force check of (for example) a million email addresses, then they'll get 100 000 positive responses, the vast majority of which will be false positives.

Re: The False Allure of Hashing for Anonymization

#98

Post by GordonS »

bglusman wrote: post #81
Area12 wrote: post #74

I am not a crypto expert, but I thought that the idea was to produce a new more or less random salt for EACH password, store the salt with the hashed password, hashing using an expensive algorithm. Yes the hacker steals the salt with the hash, but now has to go to the trouble of brute forcing that ONE password with its UNIQUE (or almost unique) salt. In other words, the hacker can crack it, but the process is so expe…

For the specific use case in question, what I've been doing for years is not just hashing the data, but hashing an internal secret AND the data. The secret isn't stored in the database anywhere (usually an env var but could be a secret in vault or other outside config), so our hashes are deterministic (and don't need a seperate salt for each one), but our hashes will never cooincide with another system's hashes. I di…

> hashing an internal secret AND the data

This is essentially how HMAC works (HMAC is mentioned in the article). It's generally considered safer to use a 'real' HMAC algorithm instead of rolling your own.

Re: The False Allure of Hashing for Anonymization

#99

Post by your-nanny »

kevin_nisbet wrote: post #83
your-nanny wrote: post #80

Earlier quoted context omitted.

Would someone explain to me why this is true (or not).

OP Here. With my understanding of bcrypt, it's an algorithm that's designed to be slow (and the implementation providing guarantee's to resist attempts to significantly speed it up), and the slowness is tuneable through a work factor. So usually you would target something like 200-500ms. Long enough to be slow if you have to make billions of guesses, but still fast enough that when someone enters their correct passwo…

I see, so assuming the salt is known (which I guess is a reasonable assumption in a large fraction of cases).

Re: The False Allure of Hashing for Anonymization

#100

Post by nebulous1 »

PeterisP wrote: post #97
nebulous1 wrote: post #85

Earlier quoted context omitted.

I'm not sure you could make the data statistically meaningful and have too many false positives to deanonymize an id. I think you're basically suggesting randomly grouping the ids so they average X real ids per grouped ID. At least if you just did it randomly instead of by hashing then there would be no danger of a dictionary attack.

The expectation is that a brute force attack would try orders of magnitude more IDs than you actually have. It means that if a random ID is 90% likely to have a unique hash and 10% likely to map to one of your real IDs, then your real data won't have that many collisions, however, if someone does a brute force check of (for example) a million email addresses, then they'll get 100 000 positive responses, the vast majo…

That's a reasonable point but doesn't explain why you're using hashes instead of random groupings in the first place.
Post reply on HN