The City of Seattle Accidentally Gave Me 32M Emails for $40 • Hacker News Board

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#111

Post by tvanantwerp » Sun, Oct 21, 2018, 12:51 PM UTC

Tempest1981 wrote: post #44

daenz wrote: post #41

Somewhat related, I'm constantly shocked (maybe I shouldn't be anymore) at the tech ineptitude of cities that are supposed to be big tech hubs. I live in Seattle, and my regular tech complaint is we can't get the buses connected to an app that is accurate within +-10 minutes. I know it doesn't sound like much, but how much tech brainpower is here, and why isn't that tech shining more clearly?

Because private tech companies fight like mad to avoid paying taxes to public city operations? Citizens too, probably. Nobody likes taxes. And the best/brightest tech workers gravitate to the higher private salaries. It’s not a technical problem - those are easier.

I live in DC, land of the professional Fed. At the absolute highest level and after adjusting for location, the most a DC Fed could earn is $164,200. No surprise that anyone with serious technical talent--and by extension, market value--doesn't want such a job.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#112

Post by sverhagen » Sun, Oct 21, 2018, 12:56 PM UTC

jacquesm wrote: post #47

bpchaps wrote: post #42

Earlier quoted context omitted.

I probably only spent 30 minutes looking at it and used a few regular expressions to look for anything interesting. The point there was to understand the extent of the leak so that I could raise it in the intent of being taken seriously. A search for "(Fuck|Shit|Bitch)" can go a long way. For what it's worth, I used to work at an investment bank spending 30hr/week diving through logs with unix tools, so finding inter…

FWIW I think you should not have done that, though I understand the temptation. At the first indication that the data was not what you requested and contained more than you - or they - bargained for you should have stopped looking at it and alerted both the sender and the relevant data protection authorities in so far as those are a functioning entity where you live to tell them they have an 'accidental disclosure' o…

Just curious, are there established guidelines that are broadly accepted and that lay out how to proceed? Or is it really just down to the "I think you should" on Hacker News? (No snark intended here.)

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#113

Post by nickthemagicman » Sun, Oct 21, 2018, 1:02 PM UTC

Mental note to self.

Instead of reporting data breaches turn it into a torrent and make it public.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#114

Post by [deleted] » Sun, Oct 21, 2018, 1:10 PM UTC

united893 wrote: post #14

I find the writer to be a bit of a dick in his responses. Yes, the city IT may not be at the same level as Google engineers, but there’s no need to mock their ballpark estimates, and after the mistake there’s no need to be a jerk about it. Be forthright about the error. Consider being on the other side of this, due to a careless mistake the data for many people is exposed on a random strangers hard drive. Asking for…

[deleted]

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#115

Post by stordoff » Sun, Oct 21, 2018, 1:21 PM UTC

tgsovlerkhgsel wrote: post #92

c3534l wrote: post #48

Earlier quoted context omitted.

I have never assumed that an email address I gave the government would be protected. I would also not assume that the contents of any email I sent would be in any way protected either. The government is collectively owned. Your police record, where you live, who you're married to, and whether or not you voted last election are publicly available. I would rather all of that be protected in some way, but I think it's c…

> Your police record, where you live, who you're married to, and whether or not you voted last election are publicly available. This is highly country specific. For the marriage record, I checked the laws in Germany, and (except for your own records) you have to present a "legal interest", which seems to be stricter than a "legitimate interest" (i.e. probably you need the information to enforce your rights, not just…

In the UK, as far as I know of those only marriage records is open. Police records can be obtained by your employer, but even then most minor (sentence less than 4 years) offenses are eventually considered spent and not disclosed to most roles. The electoral role (addresses) is open by default, but you can opt-out (though can still be used for certain narrow purposes), and as far as I know there is no way to check if someone voted (I've never heard of it happening, and searching doesn't give any information about it)

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#116

Post by Jakob » Sun, Oct 21, 2018, 1:28 PM UTC

jacquesm wrote: post #27

Interesting dataset. Data like this can be used to identify strong links between contractors and government officials. One problem is that the metadata should have only contained anonymized entries for the email addresses of the counterparties of the Seattle.gov addresses, the article leaves this unclear. Another potential problem is that if a case of corruption or nepotism is identified that has not been passed to t…

The linked Kaggle dataset https://www.kaggle.com/foiachap/seattle-email-metadata/ shows that the final returned data are Excel tables with content which looks like this:

  Sender or Created by: "Herring, Kaya" 
  Recipients in To line: Ortiz, Piper; Jones, Raphael
  Recipients in Cc line: Valdez, Khloe
  Recipients in Bcc line: 
  Sent: 3/23/17 18:08

(I changed the names to random ones)

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#117

Post by saosebastiao » Sun, Oct 21, 2018, 1:33 PM UTC

jacquesm wrote: post #27

Interesting dataset. Data like this can be used to identify strong links between contractors and government officials. One problem is that the metadata should have only contained anonymized entries for the email addresses of the counterparties of the Seattle.gov addresses, the article leaves this unclear. Another potential problem is that if a case of corruption or nepotism is identified that has not been passed to t…

But why would we need to protect identities of public servants? I disagree with the entire presumption that the information requested is private. It falls squarely within the principles of the laws on the books and is consistent with the ideals of open government that led to those laws being instituted in the first place.

More importantly, information that can be used to blackmail a public servant is IMO information that should be kept public. Blackmail is only useful if that information is kept private between a blackmailer and victim. Put it all on WikiLeaks and suddenly blackmail holds no weight because the blackmailer lost his leverage. If the information is of public interest and is severe enough that it is blackmail worthy, the entire public deserves to know it.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#118

Post by buckminster » Sun, Oct 21, 2018, 1:43 PM UTC

jacquesm wrote: post #47

bpchaps wrote: post #42

Earlier quoted context omitted.

I probably only spent 30 minutes looking at it and used a few regular expressions to look for anything interesting. The point there was to understand the extent of the leak so that I could raise it in the intent of being taken seriously. A search for "(Fuck|Shit|Bitch)" can go a long way. For what it's worth, I used to work at an investment bank spending 30hr/week diving through logs with unix tools, so finding inter…

FWIW I think you should not have done that, though I understand the temptation. At the first indication that the data was not what you requested and contained more than you - or they - bargained for you should have stopped looking at it and alerted both the sender and the relevant data protection authorities in so far as those are a functioning entity where you live to tell them they have an 'accidental disclosure' o…

If someone accidentally sends me information I owe them no duty of confidence. I'm under no obligation to notify them. It is entirely their problem.

The idea that the OP is at fault for looking at data which the city had already published has no basis in law.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#119

Post by techsupporter » Sun, Oct 21, 2018, 1:56 PM UTC

TeMPOraL wrote: post #46

daenz wrote: post #41

Somewhat related, I'm constantly shocked (maybe I shouldn't be anymore) at the tech ineptitude of cities that are supposed to be big tech hubs. I live in Seattle, and my regular tech complaint is we can't get the buses connected to an app that is accurate within +-10 minutes. I know it doesn't sound like much, but how much tech brainpower is here, and why isn't that tech shining more clearly?

Because the problem isn't tech, but business models. No amount of tech brainpower can help when bus operators think of their schedules as data to be sold instead of given away for free.

At least in Puget Sound, every transit operator (except for Community Transit) makes their real-time bus arrival data publicly available for free via Sound Transit's Open Transit Data service, OneBusAway: https://www.soundtransit.org/Open-Transit-Data

(OneBusAway is the name of the app that shows information to end users and the name of the API service that developers can query.)

I know that this service is widely available and free because I've had an API key for years and have (accidentally) pummeled the service with requests and have heard nary a peep from Sound Transit asking me for money.

Re: The City of Seattle Accidentally Gave Me 32M Emails for $40

#120

Post by HeadInTheClouds » Sun, Oct 21, 2018, 1:56 PM UTC

marklyon wrote: post #108

This could have been an extremely valuable dataset for the legal community. The Enron data is currently guiding much of our machine learning validation, simply because it's available.

As a general point, I totally agree with this. The Enron dataset released over ~15 years ago is still used by EDiscovery and other legal vendors along with other researchers.

There have been a huge number of papers using this dataset and there are not many other datasets of its type or size available and despite its age is one of the best we have. If people are aware of legally released datasets with a similar size and content I would be interested to hear about them.