Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The False Allure of Hashing for Anonymization

gravitational.com

Post reply on HN
Print view View original

81–90 of 106 posts

Re: The False Allure of Hashing for Anonymization

#81

Post by bglusman »

Area12 wrote: post #74

I am not a crypto expert, but I thought that the idea was to produce a new more or less random salt for EACH password, store the salt with the hashed password, hashing using an expensive algorithm. Yes the hacker steals the salt with the hash, but now has to go to the trouble of brute forcing that ONE password with its UNIQUE (or almost unique) salt. In other words, the hacker can crack it, but the process is so expe…

For the specific use case in question, what I've been doing for years is not just hashing the data, but hashing an internal secret AND the data. The secret isn't stored in the database anywhere (usually an env var but could be a secret in vault or other outside config), so our hashes are deterministic (and don't need a seperate salt for each one), but our hashes will never cooincide with another system's hashes. I didn't see this mentioned in article but didn't read thoroughly, I thought this was a pretty good compromise but curious for other perspectives/forget if I read this technique somewhere or just made it up as a reasonably good safeguard.

Re: The False Allure of Hashing for Anonymization

#83

Post by kevin_nisbet »

your-nanny wrote: post #80
rossy wrote: post #79

Earlier quoted context omitted.

This is addressed in the article under "What if we make the hash slow?" You might be tempted to use salted hashes, but apparently this only works for protecting data that is supposed to be unpredictable, like passwords, and it's not too much of a setback if the data is easily predictable, like if it's a username or email (or presumably, an IP address or a timestamp.) > Even with something like bcrypt at reasonable wo…

Would someone explain to me why this is true (or not).

OP Here.

With my understanding of bcrypt, it's an algorithm that's designed to be slow (and the implementation providing guarantee's to resist attempts to significantly speed it up), and the slowness is tuneable through a work factor. So usually you would target something like 200-500ms. Long enough to be slow if you have to make billions of guesses, but still fast enough that when someone enters their correct password they're not sitting around waiting for you're login to complete.

If my database is something like:

  salt1 + alex = aaaaaaa
  salt2 + ben = bbbbbbbb
  salt3 + kevin = ccccccc
* with 100,000 entries

This means if I want to find user = kevin in this database, I take the salt + username, and test if it produces a match in the database. Once I get to salt3 + kevin, and produce cccccc, and see that cccccc is in the database, I've now unmasked that user.

At 500ms, a single CPU can test 172,800 entries per day, which could easily scan the database. If it's millions, or tens of millions of users, you do need more resources, and if you want to unmask every user it will take some time, but it becomes plausible for a moderately sophisticated adversary with moderate resources.

The algorithms like bcrypt, scrypt etc are great for passwords, where depending on the password used, you need to make billions of guesses. However, if you reduce the problem space down to millions or thousands of guesses, because you already know the username, the email address, or the date of birth because this is not secret information but public information, these algorithms being slow helps significantly, but not enough to work alone.

Re: The False Allure of Hashing for Anonymization

#84

Post by jack9 »

js8 wrote: post #77
polskibus wrote: post #51

Earlier quoted context omitted.

Please elaborate on whether hashing is a good pseudonimization strategy in context of GDPR guidelines.

I thought the blog post made it pretty clear that it isn't.

The blog post wasnt about

> pseudonimization strategy in context of GDPR guidelines.

i.e. Doing the minimal work to meet the guidelines. Effectiveness is incidental to that goal. Maybe I misunderstood the GP tho.

Re: The False Allure of Hashing for Anonymization

#85

Post by nebulous1 »

PeterisP wrote: post #36

SHA256 pretty much ensures that you have a unique hash for every value - and that's a feature you don't want for anonymization. So why not simply take the first few bytes of a SHA256, a small enough set to ensure that collisions not only might happen but will happen? I mean, that's a required feature to ensure anonymization, not just pseudonymization - if you can select a whole trail of events for ID #123 and be sure…

I'm not sure you could make the data statistically meaningful and have too many false positives to deanonymize an id. I think you're basically suggesting randomly grouping the ids so they average X real ids per grouped ID. At least if you just did it randomly instead of by hashing then there would be no danger of a dictionary attack.

Re: The False Allure of Hashing for Anonymization

#86

Post by Nomentatus »

unilynx wrote: post #75
merinowool wrote: post #61

Earlier quoted context omitted.

It doesn't say that information cannot be _reasonably_ reconnected, but that you shouldn't be able to reconnect it at all. I don't know how you have drawn that it shouldn't be NSA-proof from this text if it literally says "in such a manner that the data subject is not or no longer identifiable."

Its in the original link, I may have limited the quote too much: ... To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used ...

Thanks for the quote. Wow. I wonder if this odd definition doesn't render "unidentifiable" to mean "almost certainly identifiable by someone, with a current technique" - since, given enough techniques, most of them will be statistically unusual. I admit it's a start, but mangling semantics that baldly gives me the willies.

The parallel history of cryptography is little more than a history of overconfidence re what counters were thought to be likely, and not. Do we really need to recapitulate that?

Re: The False Allure of Hashing for Anonymization

#87

Post by Nomentatus »

GordonS wrote: post #48
JumpCrisscross wrote: post #45

Earlier quoted context omitted.

> If you can't connect it to the user in any way, it's no longer personal information Just because you can't connect it doesn't mean nobody else can.

For all practical purposes, a secure, one-way cryptographic hash is irreversible.

I think you need to address converse the examples in the article in order to assert this.

Re: The False Allure of Hashing for Anonymization

#88

Post by pcarolan »

aidos wrote: post #62

I saw a case a few years ago where the management of a company I knew were worried that the sales team were covering their mistakes and lying about it to blame the (external) dev team's code. They asked me to take a look into it one morning. At first glance there didn't seem to be a lot to go on. There was no auditing in the application itself so I focused on the nginx logs. It's amazing how clear of a picture you ca…

I think the bigger problem was the culture that company had in place that would lead people to do that.

Re: The False Allure of Hashing for Anonymization

#89

Post by kahnjw »

awesomepantsm wrote: post #82
procrastinatus wrote: post #4

Differential privacy seems like a pretty good approach to this problem. https://machinelearning.apple.com/2017/12/06/learning-with-p...

Differential privacy is basically a buzzword. Don't believe the hype.

It seems to me that diffpriv is a nascent area of research that has not yet been bastardized by the business community. The complete opposite of a buzzword.
Post reply on HN