A self-inflicted data breach • Hacker News Board

Re: A self-inflicted data breach

#11

Post by CodesInChaos » Sun, Apr 17, 2022, 2:40 PM UTC

> This blog post is not about Gravatar.

Stackoverflow used those MD5 hashes to display gravatars. Downloading old stackexchange data dumps is simply an easy way of collecting stackoverflow's gravatar hashes.

> Some time in 2013 the email hash issue was brought to light and Stack Exchange promptly removed the email hashes from their dumps, but the damage was already done.

The issue has been known since at least 2009: http://www.developer.it/post/gravatars-why-publishing-your-e...

And I warned of this exact issue on meta.stackexchange in 2011: https://meta.stackexchange.com/a/84734/152255

In 2011 the dump contained 95k unique email hashes plus 10k IPv4 addresses, while the dump mentioned in the article had grown to 1.8M.

Stack exchange ignored the issue for several years, instead of fixing it promptly. This delay increased the number of affected email addresses increased 20x.

Re: A self-inflicted data breach

#12

Post by kingcharles » Sun, Apr 17, 2022, 5:59 PM UTC

shkkmo wrote: post #3

sneak wrote: post #2

What is the basis for the claim that md5 hashed addresses are easily reversed? You can of course hash public email lists but that won't work for unique emails.

MD5 was intended to be a message digest hash, so its primary function was to provide non-collidable checksums. (Edit: which it does poorly in an adversarial environment so is now retired for security uses.) Unlike hashing algorithms designed to be hard to reverse, MD5 has a very fast algorithm, so rainbow tables are fast and easy to create. A public list of emails only speeds up the process and is not required as you…

I wonder if there are two "sane" looking email addresses that collide on the same MD5 hash?

Re: A self-inflicted data breach

#13

Post by CodesInChaos » Sun, Apr 17, 2022, 7:45 PM UTC

kingcharles wrote: post #12

shkkmo wrote: post #3

Earlier quoted context omitted.

MD5 was intended to be a message digest hash, so its primary function was to provide non-collidable checksums. (Edit: which it does poorly in an adversarial environment so is now retired for security uses.) Unlike hashing algorithms designed to be hard to reverse, MD5 has a very fast algorithm, so rainbow tables are fast and easy to create. A public list of emails only speeds up the process and is not required as you…

I wonder if there are two "sane" looking email addresses that collide on the same MD5 hash?

It's very likely they exist. But I don't think the existing attacks on MD5 will allow you to construct one. So you'd have to spend 2^64 on brute-force, which is expensive but feasible.