Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The City of Seattle Accidentally Gave Me 32M Emails for $40

mchap.io

Post reply on HN
Print view View original

81–90 of 239 posts

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#81

Post by jacquesm »

rocqua wrote: post #75
jacquesm wrote: post #27

Interesting dataset. Data like this can be used to identify strong links between contractors and government officials. One problem is that the metadata should have only contained anonymized entries for the email addresses of the counterparties of the Seattle.gov addresses, the article leaves this unclear. Another potential problem is that if a case of corruption or nepotism is identified that has not been passed to t…

hash@hash alone isn't enough. Keyed hashes, with a secret key might work. The issue with hash@hash is that it is still possible to see whether a given person sent an email. Moreover, there are probably similar issues as with hashed_known_hosts as described in [1]. In short, the space of possible emails might be small enough to just brute-force search for all e-mails. [1] https://news.ycombinator.com/item?id=18082033

> The issue with hash@hash is that it is still possible to see whether a given person sent an email.

In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses.

Even just seeing the graph laid out would allow you to infer who some of the players are. In general, to release such information on the assumption that it will be impossible to reverse it is irresponsible, and I would have loved for the city to recognize this and to get a judge to sign off on the release.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#82

Post by giancarlostoro »

nickysielicki wrote: post #38

> Seattle was approaching the problem as if they were pursuing Computer Fraud And Abuse (CFAA) charges. For information that they sent. Jiminey Cricket.. > So, I deleted the files. Isn't it great to live in a country where we have generic felonies that governments can apply to just about anything involving a computer and ruin your life? Land of the "free" and the home of the 'fraid.

Where police treat you as fully guilty if you're a suspect despite you having to be thought of as innocent until proven guilty. A number of people have gone to jail because they knew not to keep their mouths shut and told cops too much that made them sound like criminals and guilty. Most people don't know where they were 2 hours ago, let alone the night of December 5th, 1957 at 2:05 AM, like SERIOUSLY?

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#83

Post by PakG1 »

mjw1007 wrote: post #80
PakG1 wrote: post #67

Earlier quoted context omitted.

The vendors may be horrible, but it's still on the shoulders of the city for choosing the vendors. This is really down to the fact that most decision makers have no idea how to understand or differentiate between options. It's been like this for decades. When a secretary of state doesn't know the risks in running a private independent email server and how to ensure those risks don't become issues, how do expect much…

I don't think it's good enough to say « it's still on the shoulders of the city for choosing the vendors ». If I write a piece of software which is technically capable of meeting its requirements if you read the manual carefully enough, but in practice the intended users can't figure out how to do so, that piece of software is no good. Similarly if the market is in principle providing IT vendors who are capable of pr…

Do you have a viable method to fix the problems you describe? If not, then we're still left with it being on the shoulders of the city. Ultimately, it's really hard to police that the vendors don't make crap solutions. If the market fails to figure out which are crap, the market may have failed, but I don't have any ideas that would succeed better.

Edit: and let me say that I posited that the issue isn't market forces. The issue is lack of expertise at decision making levels. Even if there were zero market, it wouldn't stop people from doing it wrong.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#84

Post by rocqua »

jacquesm wrote: post #81
rocqua wrote: post #75

Earlier quoted context omitted.

hash@hash alone isn't enough. Keyed hashes, with a secret key might work. The issue with hash@hash is that it is still possible to see whether a given person sent an email. Moreover, there are probably similar issues as with hashed_known_hosts as described in [1]. In short, the space of possible emails might be small enough to just brute-force search for all e-mails. [1] https://news.ycombinator.com/item?id=18082033

> The issue with hash@hash is that it is still possible to see whether a given person sent an email. In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Even just seeing the graph laid out would allow you to infer who some of the players are. In general, to release such information on…

> In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses.

Agreed, hence the need for more than a plain hash. Note that technically, a 'salt' is unique per user and generally doesn't need to be kept secret. It really only applies to storing passwords.

What I suggested is more like a pepper [1], but in this use-case, you could use the same pepper for every address. Alternatively, you could just generate UUIDs for each address and publish those, but that requires a lookup in the UUID table for every e-mail. (Just like salted hashes would require a lookup to the salt for every e-mail).

[1] https://en.wikipedia.org/wiki/Pepper_(cryptography)

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#86

Post by jacquesm »

rocqua wrote: post #84
jacquesm wrote: post #81

Earlier quoted context omitted.

> The issue with hash@hash is that it is still possible to see whether a given person sent an email. In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Even just seeing the graph laid out would allow you to infer who some of the players are. In general, to release such information on…

> In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Agreed, hence the need for more than a plain hash. Note that technically, a 'salt' is unique per user and generally doesn't need to be kept secret. It really only applies to storing passwords. What I suggested is more like a pepper…

I don't think you could use the same 'pepper' for every address. After all, if you know at least one address in the database (for instance, your own) and what time you sent the email (which you do) then you could use that to recover the pepper that was used for the hashing. So I really do believe the salt should be unique per address used.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#87

Post by rocqua »

jacquesm wrote: post #86
rocqua wrote: post #84

Earlier quoted context omitted.

> In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Agreed, hence the need for more than a plain hash. Note that technically, a 'salt' is unique per user and generally doesn't need to be kept secret. It really only applies to storing passwords. What I suggested is more like a pepper…

I don't think you could use the same 'pepper' for every address. After all, if you know at least one address in the database (for instance, your own) and what time you sent the email (which you do) then you could use that to recover the pepper that was used for the hashing. So I really do believe the salt should be unique per address used.

As per the wikipedia article:

" Where the salt only has to be long enough to be unique, a pepper has to be secure to remain secret (at least 112 bits is recommended by NIST), otherwise an attacker only needs one known entry to crack the pepper. "

If you use e.g. a 128 bit pepper, anyone trying to brute-force that based on a known email-hash combination would need to brute force 128 bits.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#89

Post by ehsankia »

JoeSmithson wrote: post #32

I'm very surprised they gave out this information. I'm not talking about the mistake, I mean the actual request. In the UK I don't think you could even get a production order for this. Like, it's effectively getting Communications Data simultaneously against thousands of people not suspected of any crimes?? Like, do people know that by emailing their local government their email address is now free for scammers to re…

The part I found even more strange is that people are sending their credit card numbers and other personal information through e-mail...

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#90

Post by amingilani »

I'm simultaneously impressed and saddened by how fast the responses for these FOIA requests were proceeded by the government.

And here in my country, I needed a court order to get at least an acknowledgement of my FOI request.

And now I'm petition court intervention to get the FOI processed in accordance with the law.

Post reply on HN