Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Сhecking for leaked data without storing any

medispank.com

Post reply on HN
Print view View original

11–20 of 24 posts

Re: Сhecking for leaked data without storing any

#11

Post by jayhoon »

TeMPOraL wrote: post #6
jayhoon wrote: post #3

Earlier quoted context omitted.

While that's true, the phone numbers without the name and other information cannot be weaponized. The phone numbers can be generated - there are only 8 changing digits in Australian mobile phone numbers (the first 2 digits are always 04).

> the phone numbers without the name and other information cannot be weaponized. On the contrary - they can and routinely are. Speculative case: The space of valid phone numbers is small, the total number of subscribers is peanuts for automation, and bulk calling is cheap. As a result, it's affordable in some cases to just literally call every single number (prefiltered through one of the marketer databases of number…

Calling random or consecutive phone numbers is essentially a spraying attack. That can be done without having a database of numbers. While they are annoying, they cannot be used to attack a person or a group of people without additional information. That's what I mean in saying that the numbers alone cannot be weaponized.

Australian mobile phone system has a capacity of 100M numbers, which means that with a probability of 1/4 a random phone number is a real one (Australia's population is nearly 26M).

Re: Сhecking for leaked data without storing any

#12

Post by jayhoon »

xyzzy123 wrote: post #8
jayhoon wrote: post #3

Earlier quoted context omitted.

While that's true, the phone numbers without the name and other information cannot be weaponized. The phone numbers can be generated - there are only 8 changing digits in Australian mobile phone numbers (the first 2 digits are always 04).

Then why not publish all the phone numbers in clear.

Additional benefits of randomising data by sorting the hashes, for example.

Re: Сhecking for leaked data without storing any

#13

Post by xyzzy123 »

jayhoon wrote: post #12
xyzzy123 wrote: post #8

Earlier quoted context omitted.

Then why not publish all the phone numbers in clear.

Additional benefits of randomising data by sorting the hashes, for example.

Could you explain those benefits please?

Please accept my genuine apologies here, I think I must have crossed some wires / be operating under different assumptions / be talking past you somehow.

To clearly state my position, I believe:

a) Publishing a list of sha256(.au phone number) is equivalent to publishing the list in clear

b) We are discussing the set of phone numbers of affected Medibank customers, which could be described in a variety of ways (e.g. bitmap, dumb list, hashed list, bloom / cuckoo / xor filter etc).

c) There is basically nothing efficiency wise you need to think about to provide lookups in a database of phone numbers if you don't care about security. It is "laughably small data". All Australian phone numbers will fit comfortably in an Excel spreadsheet, Sqlite database, greppable text file etc.

Finally: Phone numbers and emails are a little weird. They are PII but the seriousness of their disclosure depends on context. Imagine a publicly posted list of porn purchases or medical conditions by phone number. It has a sort of "casual privacy" about it but is quite transparent to each user's contacts.

Re: Сhecking for leaked data without storing any

#14

Post by danbruc »

xyzzy123 wrote: post #13
jayhoon wrote: post #12

Earlier quoted context omitted.

Additional benefits of randomising data by sorting the hashes, for example.

Could you explain those benefits please? Please accept my genuine apologies here, I think I must have crossed some wires / be operating under different assumptions / be talking past you somehow. To clearly state my position, I believe: a) Publishing a list of sha256(.au phone number) is equivalent to publishing the list in clear b) We are discussing the set of phone numbers of affected Medibank customers, which could…

The data is already in the wild but you do not want to spread it even more, so you do not just publish it. But you still want to enable people to find out if they are in the data set, so you offer a website to check for their data.

You could just keep the data on your server in plain text, even if it would get hacked, there would be no real additional damage as the data is already in the wild. There might however be legal reasons why you can not do this.

So as an exercise in how to handle such data properly or in order to comply with the law, you want a secure solution, i.e. assuming the data is not already in the wild, how do you implement this in a way that getting your server hacked does not leak the data?

The idea was then to just hash everything with SHA-256, but this does not work because SHA-256 is fast and the search space relatively small, so the hashes can be more or less easily reversed. The next better option would be to hash the data with something slow like bcrypt, that would make reversing the hashes orders of magnitude slower with a well chosen work factor.

haveibeenpwned.com also had an article discussing the decisions they made in quite some detail.

Re: Сhecking for leaked data without storing any

#15

Post by gorokan »

danbruc wrote: post #14
xyzzy123 wrote: post #13

Earlier quoted context omitted.

Could you explain those benefits please? Please accept my genuine apologies here, I think I must have crossed some wires / be operating under different assumptions / be talking past you somehow. To clearly state my position, I believe: a) Publishing a list of sha256(.au phone number) is equivalent to publishing the list in clear b) We are discussing the set of phone numbers of affected Medibank customers, which could…

The data is already in the wild but you do not want to spread it even more, so you do not just publish it. But you still want to enable people to find out if they are in the data set, so you offer a website to check for their data. You could just keep the data on your server in plain text, even if it would get hacked, there would be no real additional damage as the data is already in the wild. There might however be…

You are right; choice of SHA256 is about handling personal data safely.

Unfortunately, the leaked data is available to anyone. There is no point of putting an extra effort of recovering the data from its hashed form.

Re: Сhecking for leaked data without storing any

#16

Post by xyzzy123 »

gorokan wrote: post #15
danbruc wrote: post #14

Earlier quoted context omitted.

The data is already in the wild but you do not want to spread it even more, so you do not just publish it. But you still want to enable people to find out if they are in the data set, so you offer a website to check for their data. You could just keep the data on your server in plain text, even if it would get hacked, there would be no real additional damage as the data is already in the wild. There might however be…

You are right; choice of SHA256 is about handling personal data safely. Unfortunately, the leaked data is available to anyone. There is no point of putting an extra effort of recovering the data from its hashed form.

Thanks all. My "usual go to" answer for this (protecting data where you'd like to hash it but there's not really enough entropy) is in a sibling comment.

Agree, all the data being public already renders everything a bit moot.

Re: Сhecking for leaked data without storing any

#17

Post by bluesign »

xyzzy123 wrote: post #7
bluesign wrote: post #5

Earlier quoted context omitted.

not sure if there is an effective solution to this. (phone number specific)

We had to meet a similar requirement and used hardware anchoring. Instead of a hash, think of a keyed HMAC style operation where the HMAC secret is inside an HSM. HSM operations are handled by a separate team than webapp. Key is not extractable. It is not vulnerable to brute-force in the same way as a hash because the function cannot be executed outside your computing environment. Of course an attacker can still try…

doesn't "proof of ownership" kill the purpose ? ( my email/phone number not seen by the operator )

My attack scenario was: they don't have the data, harvesting my data.

Re: Сhecking for leaked data without storing any

#18

Post by an_aparallel »

this is kinda wierd...someone in my house has been affected by the leak. So i typed in their name. And it says they're not in the leak. 3 other people in my house including me have apparently all bean leaked according to this site. So, either this resource doesnt work, or medibank is not being forthcoming in a timely manner as to who is being affected. I hope they get sued to oblivion for this.

Does anyone know what one is specifically to do about their home address when it is leaked? It's easy to change emails, mobile numbers and so on...but...home addresses...not so easy.

EDIT: im guessing 1 piece of the puzzle (an address) with all the other things changed should be OK? who knows...

Re: Сhecking for leaked data without storing any

#19

Post by jayhoon »

an_aparallel wrote: post #18

this is kinda wierd...someone in my house has been affected by the leak. So i typed in their name. And it says they're not in the leak. 3 other people in my house including me have apparently all bean leaked according to this site. So, either this resource doesnt work, or medibank is not being forthcoming in a timely manner as to who is being affected. I hope they get sued to oblivion for this. Does anyone know what…

It seems the hackers published only a part of the stolen data.

The same situation here: several people on the same card, some are in the leak and some not.

Re: Сhecking for leaked data without storing any

#20

Post by an_aparallel »

jayhoon wrote: post #19
an_aparallel wrote: post #18

this is kinda wierd...someone in my house has been affected by the leak. So i typed in their name. And it says they're not in the leak. 3 other people in my house including me have apparently all bean leaked according to this site. So, either this resource doesnt work, or medibank is not being forthcoming in a timely manner as to who is being affected. I hope they get sued to oblivion for this. Does anyone know what…

It seems the hackers published only a part of the stolen data. The same situation here: several people on the same card, some are in the leak and some not.

this is so infuriating, i'm going to change all my details, but to what end? Optus, and Medibank have had leaks...so its a matter of time until this happens again - then what? i change my mobile number/email addresses...again?
Post reply on HN