Live data from Hacker News

I accidentally started a movement – Policing the Police by scraping court data

news.ycombinator.com

111–120 of 202 posts

Re: I accidentally started a movement – Policing the Police by scraping court data

#111

Earlier quoted context omitted.

It still isn't very clear what the focus and direction they want to go in is from that page. What would be an example of a core data set they are trying to compile? police involved shootings? police budgets? everything.

Seems pretty obvious from the page: > Our mission is to make data from every U.S. police agency accessible via a single public resource. More precisely: > There are over 18,000 police organizations, and each has a unique way to publish information. i.e. Police are publishing data? Let's organize it and make it easily accessible.

I am curious and want to know more.

"Police data" is incredibly vague. What types of data are most interest and available from most of every agency?

If the only answer to what kind of data is "police data" then I'm not sure if I should care, support, or contribute.

Is this data on how many toilet paper roles departments purchase or police involved shootings? neither?

Certainly you can see where the question is coming from right?

Re: I accidentally started a movement – Policing the Police by scraping court data

#112

Earlier quoted context omitted.

Because cops are lazy and want to put it all in a report later? I would.

A ton of stuff is done on mobile terminals. Computer aided dispatch including status and location of a unit, report writing, identity checks, etc. Scanners were useful because a cop would ask the dispatcher for a license check on Joe Blow, age 34, of Smithville - or registration check on a certain license plate number. These days by the time an officer flips on his lights to pull someone over, they probably already k…

This is a problem when the computer is wrong. I've been pulled over for having "unregistered" plates that were perfectly valid.

Re: I accidentally started a movement – Policing the Police by scraping court data

#113
post #95
post #79

This is important. Locally, we had a sheriff who was being heavily, heavily criticized due to several deaths at the county facility. This was at the height of the protests a few years ago. It was a lot of work to find data on policing nationwide, because the question really was "Is the sheriff doing a bad job, or do bad things happen sometimes?" After some hard work trying to identify other cities with similar socioe…

It's the same sort of thing with body cameras. If anything they capture a lot more context about situations. Generally I think the police will start to want to have the safety of the record keeping rather than not.

I sort of agree, but one thing the last few years showed me was how little people (especially in the media and parts of the academy) care about the truth. overwhelming, easily accessible, incontrovertible evidence might save you from the law, but the media, activists, and other partisans are still happy to make your life hell.

Re: I accidentally started a movement – Policing the Police by scraping court data

#114

Really love the idea, and the passion behind it. Def could have legs. Here’s the pitfalls I see you falling into: (1) seriously, what data are you collecting? “Everything” isn’t a great answer (who’s supposed to use ‘everything’, anyway? “Anyone”?). “Apples-to-apples police misconduct statistics” is a good one. (2) it’s important to clarify 1 because you need to know who you’re serving, and why. Different activists n…

Thanks for the thoughtful response! This is really helpful.

1. Agreed. Our strategy for this isn’t clear on the website, I guess, but we do have one. It’s to focus on depth in geographic areas. This is because context is critical, and because most of the users we talk to are operating locally with municipal or county level data. So it’s more important to have every data source we can possibly find relevant to Pittsburgh than it is to have every arrest record in every municipality. Or at least, it’s more immediately useful to people.

That said, most people seem to contribute data sources from where they live. I think little microcosms will spring up where people take stewardship of maintaining information about their chosen geo or subject areas. Not too far down the roadmap, Milestone 2 for the PDAP heads.

2. I will take it as a next step to make this strategy clear and say why. We want to basically allow the community to make its own to do list: what kind of question are you trying to answer? That creates a “bounty” for data which can be fulfilled by an altruistic volunteer, another member of your team, etc.

3. Yes. We’re not trying to do apples to apples comparisons of departments yet, partly because it’s so absurdly difficult and you don’t know where to start. Why would you undertake a 12 hour research project to compare St. Louis and Minneapolis incident reports if you don’t have a use case? Instead we’re focusing on what we DO know we need: complete local data, town by town / county by county.

The data we collected reflects the nature of our early experiments, which were scattered. This airtable prototype is maybe 2 weeks old, next up is helping people understand where to focus.

The idea for demonstrating value is also local. I’m working with groups in Pittsburgh (where we are based, and where our funding came from) to make ourselves indispensable to them. I’m hoping to turn the $250k into a handful of killer local case studies in this year, rather than marking 0.1% progress toward a national vision.

Thanks again for giving me the practice explaining this stuff. I hope I’m making any kind of sense, and of course happy to hear where I’m still wrong.

Re: I accidentally started a movement – Policing the Police by scraping court data

#115
post #79

This is important. Locally, we had a sheriff who was being heavily, heavily criticized due to several deaths at the county facility. This was at the height of the protests a few years ago. It was a lot of work to find data on policing nationwide, because the question really was "Is the sheriff doing a bad job, or do bad things happen sometimes?" After some hard work trying to identify other cities with similar socioe…

Tougher to answer, but maybe more useful, would be “What harm reduction strategies are being tried in other cities? Are they working?” this is at the intersection of policy and outcome and takes a lot of context.

Re: I accidentally started a movement – Policing the Police by scraping court data

#116

Earlier quoted context omitted.

We don't have any scraped data yet. I replied to the parent post addressing some of this, but mostly if people need the data they run a scraper locally and use the data that way. At the moment, our energy is going into building an app to help people submit and manage our database of data sources: https://docs.pdap.io/activities/data-sources/what-is-a-data-...

Ah very good! This sounds like it would be a perfect application of https://datasette.io/

We’ve looked at this and it’s pretty exciting! Have you used it? Know of any particularly relevant case studies we should look at?

Re: I accidentally started a movement – Policing the Police by scraping court data

#117

Earlier quoted context omitted.

Seems pretty obvious from the page: > Our mission is to make data from every U.S. police agency accessible via a single public resource. More precisely: > There are over 18,000 police organizations, and each has a unique way to publish information. i.e. Police are publishing data? Let's organize it and make it easily accessible.

I am curious and want to know more. "Police data" is incredibly vague. What types of data are most interest and available from most of every agency? If the only answer to what kind of data is "police data" then I'm not sure if I should care, support, or contribute. Is this data on how many toilet paper roles departments purchase or police involved shootings? neither? Certainly you can see where the question is coming…

Your question makes sense. I’ve addressed this elsewhere but I’m currently just answering on my phone so I’ll summarize by saying that our focus is on complete, solid local data one municipality at a time (full context is really helpful when actually using the data) as opposed to any particular type of data.

That said, we still have work to do pointing people in this direction and helping them understand why. This whole thread is going to really affect our website and docs :)

Re: I accidentally started a movement – Policing the Police by scraping court data

#118
post #96

Is there anything like this for regulatory capture in federal and state governments? I could imagine a revolving door between people working in the regulatory bodies and the industry they regulate.

I’m not aware of one. Tracking people is hard because names aren’t unique; it’s a challenge tracking officers as they move between departments, too.

Re: I accidentally started a movement – Policing the Police by scraping court data

#119
post #88

Earlier quoted context omitted.

Assuming the data is accurate it can be used to show disparities between groups for a variety of situations - traffic stops, arrests, jail vs. diversion programs, charge stacking, etc..

How is that policing the police? Are disparities supposed to be evidence of something nefarious going on? Given that there are fundamental distinctions between members in different groups (otherwise they would be in the same group) and almost certainly many other non-fundamental distinctions that correlate with the group-defining distinction, is it not entirely plausible that there should be disparities in police sta…

I will say that we are not wading into this. We focus on accessibility; if you point to a big pile of ugly data, the first thing that will happen is that a bunch of very smart people will analyze it. We’re trying to make the big pile, which is currently in like half a million small piles.

Re: I accidentally started a movement – Policing the Police by scraping court data

#120
You might try defining what the "ideal" department's data would look like: what categories of data, what columns each record has, what the values are for each, etc. Ideally you'd stamp it with a year and give it a spiffy name so it could be the National Police Data Reporting Standard 2022 (NPDRS.2022) or something.

Departments that are trying to be transparent (or who just don't want to deal with figuring it all out from scratch) may be happy to adopt something considered a "standard" for tracking and reporting data. In some cases it means it is a checkbox they can check without having to deal with annoying people and their annoying questions... but that hardly matters so long as the data is made available. It would also give companies developing software for police departments a target to aim for.

Post reply on HN