The False Allure of Hashing for Anonymization
gravitational.com
The False Allure of Hashing for Anonymization
1–10 of 106 posts
Re: The False Allure of Hashing for Anonymization
#2Author Here.
Using crypto hashes to anonymize data is one of those mistakes I've seen several times, and wanted to draw some attention to the issue so that hopefully we can all learn from it.
Let me know if you have any questions.
Re: The False Allure of Hashing for Anonymization
#3I'm surprised that there was no mention of a salt used in a secure server to generate the hashes and act as an oracle. Adding pepper at the customer site already seemed like a good idea. Of course this is still hard and requires diligence for those who care about their customers and data security.
Re: The False Allure of Hashing for Anonymization
#4Differential privacy seems like a pretty good approach to this problem.
https://machinelearning.apple.com/2017/12/06/learning-with-p...
Re: The False Allure of Hashing for Anonymization
#5tl;dr preimage resistance is only as strong as sizeof(input domain), which is probably small if you're trying to anonymize something.
Re: The False Allure of Hashing for Anonymization
#6[deleted]
Re: The False Allure of Hashing for Anonymization
#7If you don’t require deterministic hashes (and deterministic hashes are bad for anonymization anyway) just hash data+randomBytes(16) (obviously, don't save randomBytes(16) anywhere). There you are, nobody can bruteforce your hashes.
Even better, just replace your data with H(randomBytes(16)). Or a random UUID.
Re: The False Allure of Hashing for Anonymization
#8If you don’t require deterministic hashes (and deterministic hashes are bad for anonymization anyway) just hash data+randomBytes(16) (obviously, don't save randomBytes(16) anywhere). There you are, nobody can bruteforce your hashes. Even better, just replace your data with H(randomBytes(16)). Or a random UUID.
Man, that sounds a lot like "developers just doin' stuff". What's better, randomBytes(16) or H(randomBytes(16))? Or random UUID? Why hash the random bytes but not that? What are the entropy implications? Why attach an identifier to something that's supposed to be anonymous? etc etc
Re: The False Allure of Hashing for Anonymization
#9Author Here. Using crypto hashes to anonymize data is one of those mistakes I've seen several times, and wanted to draw some attention to the issue so that hopefully we can all learn from it. Let me know if you have any questions.
Have you looked into VRFs (verifiable random functions)?
This is essentially a public key crypto version of HMACs. You have a piece of data and a blinding factor, both are committed into a single point on an elliptic curve, and blinding factor is not exposed as-is (as in case with salt for hashed content), but instead committed into a public key (a point).
The resulting point preserves the algebraic structure, which allows you to sign proofs about it, but w/o leaking the blinding factor. You can even sign a proof to a "designated verifier", which makes the proof useful only to a specific party, but anyone else wouldn't be able to trust it as it could be forged by that other party.
Re: The False Allure of Hashing for Anonymization
#10Author Here. Using crypto hashes to anonymize data is one of those mistakes I've seen several times, and wanted to draw some attention to the issue so that hopefully we can all learn from it. Let me know if you have any questions.
So one of the issues here is using an externally visible ID (or a transformation of such) as an internal ID. Why not create a random int64 at account creation time which is invisibly linked to the public username (eg, email address). So now you've got a proper join key, you can restrict access to the map, and it's easy to delete the map entry when the user unsubscribes.
(There can still be good reasons to apply one-way hashing to the random internal UUID, as well: for example, to provide different levels of logs access to different internal users. People who make dashboards get hashed ids, and people who debug logging get raw ids.)
The problem of entropy allowing individual user identification even with all IDs scrubbed is still very real, though, and non-trivial to undertake. One can start by wrapping the query engine with a service which checks that a certain minimum number of people are covered by a given query before returning the results. Or apply differential privacy-type transformations to the output...