Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The City of Seattle accidentally gave me 32M emails for $40 (2018)

mchap.io

Post reply on HN
Print view View original

61–70 of 230 posts

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#61

Post by ralferoo »

legitster wrote: post #16
froh wrote: post #4

wow! in Europe, this request 1. From address 2. To address 3. bcc addresses 4. cc addresses 5. Time 6. Date correlates person-related information (who was in contact with whom at which date and time). storing it, let alone processing it is only admissible on a need to know basis. even if you jump through the hoop of an officer acting on behalf of Seattle is not a person any more, which is a stretch already, even then…

So, email addresses kind of suffer from the same problem as Social Security numbers - they were never designed to be private information! But they have been forced to fit that role. And, you know, cities have no problem giving out personal information all of the time. Your personal address, or whoever owns property in a city, is a matter of public record. And so the city has no problem handing out your name and addre…

The issue isn't that email addresses are private information, it's that they are personally identifiable.

[EDIT: to be clear, the following refers to the GP comment referring to in Europe, where as the parent comment is clearly US-centric]

Even if you replaced every distinct e-mail address with an ID or hash or whatever, they would still fall foul of GDPR because that ID or hash would become PII. You might say that's impossible, but for instance, if you knew that you happened to send specific e-mails to people in that dataset, you could correlate the times to discover your own hash, then filter by your hash and use the times to correlate with who you sent them to. Now you know the hashes of everyone you corresponded with, and can do PII-based analysis on that, e.g. how many people e-mailed them and when, whether they responded, who else they e-mailed after receiving those e-mails (so you might be able to start identifying colleagues etc), etc.

PII, even in what looks to be anonymous data, is very valuable to people who are determined to mine it, which is why typically data such as this should only be disclosed in aggregate, and usually after verifying that each aggregated set of data actually covers a large enough group of individuals that also can't be correlated another way.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#62

Post by legitster »

0cf8612b2e1e wrote: post #49
legitster wrote: post #37

Earlier quoted context omitted.

> They thanked me for bringing the situation to their attention and all that, but the mood of the call was as if both parties had a knife behind their back. Somewhere towards the end of the call, I asked them if it was okay to keep the emails. Why not at least ask, right? >...This isn't something I'm even remotely cool with, so we ended the call a couple minutes later, and agreed to have our lawyers speak going forwa…

>… and refused to cooperate with the third party auditor Well, yeah. No way I am letting someone scan and archive my drives/data to correct their mistake. They broke other people’s privacy and now they want to break mine? Pound sand. Even if they could prove my drives had been wiped, that would do nothing to prove it had not been otherwise copied elsewhere.

This is not how the law works. If enriched uranium shows up at your door, you don't just get to hold onto it just because someone screwed up.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#63

Post by ender341341 »

oh_sigh wrote: post #12

What happens if he refused the lawyer request to scan his hard drive?

IANAL but the city could sue, whether or not they would would win IDK, if I were fighting it I'd argue along the lines of "if I meant to misuse the data I wouldn't have told you I had it and scanning my hard drive opens up my personal data to the same potential mis-distribution that I notified the city of."

I agree with the author's reaction though, and any non-court ordered scanning of my system would have to happen under my supervision, with strict agreement on the how.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#64

Post by jackson1442 »

andersrs wrote: post #59

Companies and governments will always undermine privacy either by incompetence or because it's good for business. In every case the only party that benefits from more privacy is really the end user. So what can we as users do about this? Is there an email service that can generate unique alias addresses? just.for.bloated.govt.dept.123@xyz.com would really help but I'm guessing the big companies like Google and Micros…

Most self-hosted email lets you set up a wildcard address and blackhole or tag mail send to particular addresses at will. Fastmail has a 1Password integration that automates this and lets you use @fastmail.com as the domain, but using a wildcard with any provider isn’t much more difficult.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#65

Post by pierat »

The real point of the story:

Security researchers get FUCKED, hard. They're treated as evil bad hackers who will destroy everything.

Why? Cause they made people look bad.

The real answer? It was provided as public records. So slap-em into public. Fuck 'em.

(Or, sell it to a gray hat data broker and get paid.)

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#66

Post by oh_sigh »

refulgentis wrote: post #18

[flagged]

Should it be that easy to socially engineer city employees with access to huge troves of potentially confidential information?

If anything, OP did the city a favor by going out of his way to inform them. What if OP was a black hat?

Also, who cares why he wanted the data? It's his right to it as a citizen. Just like it's your right to drive around at 2am without answering to the cops, even if they find it weird.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#67

Post by _Algernon_ »

flashback2199 wrote: post #31
legitster wrote: post #27

If you can't load the blog: - He FOIA'd all metadata of emails to and from the City of Seattle. - The city IT department pushed back, saying that their policy was to hand-review each email for privacy, and this was 32m emails. - They later acquiesced and just dumped all of the meta-data into files and sent it over - They didn't realize email-preview was also meta-data, which included the first 256 char of each email.…

> The job seems awful How hard can it be to do a sanity check of the metadata?

Just as hard as not deleting a production database, yet that still happens.

This is an edge case for which they probably didn't have an existing process which means they had to wing it.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#68

Post by pierat »

kelnos wrote: post #45
legitster wrote: post #37

Earlier quoted context omitted.

> They thanked me for bringing the situation to their attention and all that, but the mood of the call was as if both parties had a knife behind their back. Somewhere towards the end of the call, I asked them if it was okay to keep the emails. Why not at least ask, right? >...This isn't something I'm even remotely cool with, so we ended the call a couple minutes later, and agreed to have our lawyers speak going forwa…

I would refuse to cooperate with the third party auditor as well. No one gets to scan my hard drive without a court order. And even then, I wouldn't provide the decryption passphrase unless I was legally required to. Even though there isn't anything illegal or incriminating on my hard drive (as far as I know, anyway), I wouldn't agree to let someone violate my privacy for a mistake they made. Regardless, this whole t…

A simple affidavit under perjury should absolutely suffice.

How do you know I didn't upload it to S3, some torrent, IPFS, or elsewhere?

I'd tell them to fuck off if they want to lay hands on my computer.. or at least come back with a court order.

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#69

Post by eschneider »

bradley13 wrote: post #11

Government IT is famously expensive, and so often a disaster. I recently needed to open an account with a local agency. The fun started with two date fields on the web form, which turned out (trial and error) to require different formats. I finally received my credentials, but they didn't work. Assuming that the password was likely the problem, I tried the password reset function, only to get a 404 error. The person…

Oof, when I started my new job this year, my I-9 form was rejected with what turned out to be two date fields with different formats. That took some debugging on my part to figure out. :/

Re: The City of Seattle accidentally gave me 32M emails for $40 (2018)

#70

Post by kelnos »

refulgentis wrote: post #18

[flagged]

> To do that right would be a full time project for a handful of people for at least months, not "a one line powershell command"

Huh? Maybe bash and not powershell, but it should be pretty simple. If they have to go into the tape backups for that data, ok, perhaps not, but a reasonable mail system should make this a fairly simple task. Even if it takes a 30-line python script, that's not a big deal, and it can be left to run unattended over how many hours or days it needs to in order to go through that many files.

> You then harangued them for charging too much -- they had to come up with an estimate, and they estimated 30 seconds to review the data they sent you for each email.

The city later admitted that its original estimate was completely wrong, because they didn't have to review the email data at all (since bodies weren't to be included). I think "harranguing" them (which there's no evidence the OP did) seems pretty reasonable when they wanted to charge tens of millions of dollars to do something that ultimately actually cost forty bucks.

Also understand that this is in general how FOIA is: people ask for information, and government agencies do whatever they can to get out of providing that information, regardless of their legal requirement to do so. It's entirely reasonable to look at initial objections from the agency with a cynical, skeptical eye.

> You then insisted they __agree not to review the data__ so you could get it cheap

Huh? The data did not legally require review. The city made a mistake initially, or intentionally lied in order to get out of fulfilling the request.

> You should have informed them, deleted the data, and moved on with your life.

That's... exactly what he did? His initial beating-around-the-bush response was designed with the hard-earned knowledge that people often get punished for reporting mistakes of this nature. Which... is what actually seemed to be happening after the city was actually aware of their self-inflicted data breach. They wanted him to submit to a third-party audit (no, screw that) and seemed to be moving in the direction of filing charges against him before he got his lawyer involved. For their own negligence!

I have zero sympathy for the city here. Regardless of why OP wanted this data or whether or not you or I think it's useful data for a citizen to have, the city screwed the pooch here, and OP's karma slate seems pretty clean to me, at least for this particular incident.

Post reply on HN