Live data from Hacker News

I accidentally started a movement – Policing the Police by scraping court data

news.ycombinator.com

71–80 of 202 posts

Re: I accidentally started a movement – Policing the Police by scraping court data

#71
Apologies for my ignorance but how is this going to police the police? I read the original blog post, there was lots of inferences/could and might be's/etc made but little in the way of proof of anything. What's to stop the police saying it was just circumstance that provided your results?

I'm not here defending the police, or denigrating the project, just playing devils advocate. What happens if the police just ignore you?

Re: I accidentally started a movement – Policing the Police by scraping court data

#72

Apologies for my ignorance but how is this going to police the police? I read the original blog post, there was lots of inferences/could and might be's/etc made but little in the way of proof of anything. What's to stop the police saying it was just circumstance that provided your results? I'm not here defending the police, or denigrating the project, just playing devils advocate. What happens if the police just igno…

The goal of projects like this (I have no contact with this one) is usually to convince politicians and/or the public of their results, and those groups are the ones to actually push change.

Re: I accidentally started a movement – Policing the Police by scraping court data

#73

Earlier quoted context omitted.

1. I replied to the parent comment here; our answer to the scale problem is to recognize that people doing web scraping are as decentralized as the police. Our goal is to empower people who have questions about the police to answer them. 2. You can run them locally. We're not running the scrapers anywhere, or storing extractions anywhere. 3. This is a big, big question. Right now, the answer is dependent on the use c…

Not to be too rude/negative/mean - but it seems like a big concern to me that a "police data" project that's existed for three years, has a couple employees, has some funding, etc - doesn't really seem to have any police data. If I wanted to write a scraper to gather documents off of police websites and run it by myself and store my results locally - I could. What does your project add? What I would expect to see is…

It's ok, this feedback is how we understand what our work looks like to other people, and how we improve!

We've only had paid staff for about 6 weeks. We've run several experiments and started from scratch a few times over the years. We've been slowly inching the project forward in our spare time; I was the only volunteer for much of that time, and I can't even code!

1. Sounds great, we're building something like this.

2. Even making a list of police departments is a big challenge. We've made a good start but have work to do.

3. Yep, soon we'll have something to query.

Re: I accidentally started a movement – Policing the Police by scraping court data

#75

Apologies for my ignorance but how is this going to police the police? I read the original blog post, there was lots of inferences/could and might be's/etc made but little in the way of proof of anything. What's to stop the police saying it was just circumstance that provided your results? I'm not here defending the police, or denigrating the project, just playing devils advocate. What happens if the police just igno…

Aren't forums like this for devil's advocating, like, almost exclusively? Working as expected!

We've come a long way since that post in terms of strategy and focus. Most of that time was spent with between 1 and 3 volunteers, working a couple hours a week.

Transparency is a good goal in itself, I think. People are already using this public data, we're just trying to make it more accessible.

"Policing the police" was the original phrase used on reddit, but if you look at our website (https://pdap.io), that's not a phrase we use.

Re: I accidentally started a movement – Policing the Police by scraping court data

#76

On the back end, are you using a graph? Having done some public sector accountability stuff where the org structures themselves were obfuscated, graphs and a clear data model were the decisive tech.

Can you say more about this? Feel free to reach out to my email (josh.chamberlain@pdap.io) if you'd like to share more. It sounds like you have some expertise that would be incredibly useful to us.

Re: I accidentally started a movement – Policing the Police by scraping court data

#77
post #47

I wonder if it is legal / possible to record police radio traffic and associate it with the records?

As a long time scanner enthusiast, if you actually spend anytime listening to PD radios (which is legal and easy), you will be disappointed with how little information actually goes out over the unencrypted air - just enough to get units rolling, after that, very little, for obvious reasons.

Because cops are lazy and want to put it all in a report later? I would.

Re: I accidentally started a movement – Policing the Police by scraping court data

#78
post #2

Quoted post unavailable.

You can find the roadmap, goals, and a lot of detail on the organizational structure and the scrapers that have been written so far are here https://docs.pdap.io/activities/our-process and here https://github.com/Police-Data-Accessibility-Project

With all due respect, you really need to be able to explain the goals in a sentence or two, and then follow up with a link to the pdf. An elevator pitch, a vision statement, a clearly written user statement, all would go a long ways towards explaining why this is a worth while use of ones time and energy.

I don't know whether the goal is to find bad cops who have been fired and get hired one town over and try to prevent those hirings? Or if the idea is to find people who have been arrested and ruin their lives? Or if the goal is simply to make the data available and let people use it for whatever they wish, from deepfakes to erotic fan fiction.

Data wants to be free. The data is the end goal. Public data should be public, etc all sound great. But, I've seen the mug shot database ruin people's lives, and eventually its founders had mugshots of their own.

Scraping and publishing is one thing, but knowing the goals is even more important because it would let me know why we're scraping, what we need to scrape, how to make that available, etc.

What problem are you trying to solve? And Why?

Police data is not easily searchable? Okay, so what? What good is it to make this available? What uses could it have? Even if the goal is just to build it and see how people use it, it would be helpful to know that.

I strongly suggest you consider working backwards. It's been 3 years. What does 3 years from now look like? What is the press release? How do you know you were successful, what does that mean?

Availability is nice. It could lead to transparency. That may lead to accountability. Is that the goal? Right now this data is hard to access. If you succeed it will be easier to access. And so what? What changes in the world could come because of this? If you fail, what lost opportunity do we mourn? You gotta have some sort of easily conveyed reason for doing this. If the data accessibility alone is the end goal, that's fine, not enough for me, but at least make that clear and convince me that's worth the effort.

Re: I accidentally started a movement – Policing the Police by scraping court data

#79
This is important. Locally, we had a sheriff who was being heavily, heavily criticized due to several deaths at the county facility. This was at the height of the protests a few years ago.

It was a lot of work to find data on policing nationwide, because the question really was "Is the sheriff doing a bad job, or do bad things happen sometimes?"

After some hard work trying to identify other cities with similar socioeconomic circumstances and populations, it became clear that our local sheriff was actually better than average, and that much of the outrage was fabricated.

That's also when I learned that many people don't want to listen to statistics unless they agree with their own preconceptions.

Re: I accidentally started a movement – Policing the Police by scraping court data

#80

Hello! I'm the executive director. I have a design background, have done product management in the past, and aside from keeping the lights on at PDAP and making sure we're tax-compliant I am in a product role. I talk to people using police data, and figure out where we can add value to make the data more accessible. TL:DR; If you want to write scrapers: go for it! Run your scraper, share the results in Discord and wi…

Josh, many of your comments here are displaying as "dead." The HN FAQ[0] says: > What does [dead] mean? > The post was killed by software, user flags, or moderators. Dead posts aren't displayed by default. I suspect that it is a false positive. Maybe email the mods for help? hn@ycombinator.com [0] https://news.ycombinator.com/newsfaq.html

Thanks! I contacted them and I've been reinstated.
Post reply on HN