Two billion email addresses were exposed

Addresses can lead you to public land and mortgage records, and phone numbers can lead you to names and addressed. I assume everyone can easily find that out about me once they know my name/phone number.

> we run on Azure SQL Hyperscale

Definitely the wrong technology, and was almost certainly picked only because Troy Hunt is a "Microsoft Regional Director and MVP".

Many other technologies scale better for this kind of workload. Heck, you could ask ChatGPT to write a short C# CLI tool to process the data on one machine, you don't even need a huge box.

This kind of thing comes up here regularly on HN for problems such as duplicate password detection, leaked password filtering, etc...

After previous brainstorming sessions the general consensus was that it's really hard to beat a binary file that contains the sorted SHA hashes. I.e.: if you have 1 billion records to search and you're using a 20-byte SHA1 hash, then create a file that is exactly 20 billion bytes in size. Lookup is (naively) just binary search, but you can do even better by guessing where in the file a hash is likely to be by utilising the essentially perfectly random distribution of hashes. I.e.: a hash with a first byte value of "25" is almost certainly going to be 10% of the way into the file, etc...

It's possible to create a small (~1 MB) lookup table that can guarantee lookups into the main file with only one I/O operation of a fixed size, such as 64 KB.

Sorting the data is a tiny bit fiddly, because it won't fit into memory for any reasonably interesting data size. There's tricks to this, such as splitting the data into 65,536 chunks based on the first two bytes, then sorting the chunks using a very ordinary array sort function from the standard library.

On blob storage this is super cheap to implement and host, about 50x cheaper than Azure SQL Hyperscale, even if it is scaled down to the minimum CPU count.

> But also, how did 2 billion email addresses get exposed?

The list contains emails which have been part of some other breaches. In my domain I have 2 emails that were exposed that weren't my normal email address. One of them was a typo that I used sign up for one service which was later breached. The other one was something someone used to register to service that I have never used & that service was later breached. Those emails have never been used for anything else as far as I'm aware.

Of course judging from what posted there are likely some other services as well which were breached but wasn't noticed/published until now.

> there does not seem to be any way for _me_, the person affected, to know what password were breached

You should be using a unique randomly-generated password for each website. That way, one breach doesn't lead to multiple accounts getting hijacked AND you'll know which passwords were breached solely based on the website list. The only passwords I still keep in my head are:

  1. The password to my password manager
  2. The password to my gmail account
  3. The passwords for my full disk encryption

All of those passwords are unique and not used anywhere else. Everything else is in my password manager with a unique randomly generated password for each account. And for extra protection, I enable 2fa on any site that supports u2f/webauthn.

I used to reuse the same password for everything, and that lead to a pretty miserable month where suddenly ALL of my accounts were compromised. I'd log in to one account and see pizzas I never ordered. Then I'd open uber and see a ride actively in-progress on the other side of the country. It was not fun.

Right. Having some data leaked isn't really a boolean, leaked/unleaked. It's a list of leaks, and the implicit map betweenyl your datapoints, whether by intra or interprovider mapping

For example a forum might leak a map between your mail and a password; Implicitly your affinity for that forum's topic is also now on the public record, additionally if your posts were public but under a pseudonym, that might be now known by a sufficiently motivated attacker.

Finally this may be linked with other public datasources like your public tweets or public state records, or even other leaks.

This is why the meme about all ssn's being leaked or about a list of all valid phone numbers is so asinine.