It would be nice to see a summary of the projects and their effects after finishing the quiz. By that time, I had already forgotten about some and wondered what the answers were.
[deleted]
Show HN: Guess which government programs work
51–60 of 60 posts
Re: Show HN: Guess which government programs work
#52I was thinking about why this website triggered my skeptic alarms and came up with a few things. 1. The web interface makes it hard to go back. The first thing I want to do when I see a surprising answer is reread the question. I understand this is likely just bad engineering, but it makes the whole site feel less trustworthy. 2. Some questions get summarized poorly. For example, the mindfulness question asks "What e…
There is also an unnecessary "are you sure?" notification that pops when you leave midway through the quiz.
Re: Show HN: Guess which government programs work
#53Earlier quoted context omitted.
Hey, this is really neat. It reminds me of the lesswrong.com community, where the goal is to figure out ways of offsetting cognitive biases. One thing I've always wanted to see is a mandatory A/B test built into any new legislation. That is, funding for a study after X years of the new law or program being in place would have to be part of the law. It could even including a threshold of the desired controlled effect…
For certain types of legislation the A/B test itself would create a perverse incentive for opponents of the legislation to make the negative effects occur to get the legislation revoked. For example, if you said "we're going to put twice as many cops on patrol, but we'll revoke this if crime rates don't decrease" then a rational criminal would commit more crime, in order to get the law revoked. Or if you said "we're…
It is not in the criminals best interest to commit more crime. One criminal is probably only responsible for .0001% of the crime rate in any given city. So the incentive to commit more crime to revoke the law is infinitesimal. The disincentive created by the increased risk of incarceration would completely overwhelm the incentive to create more crime.
This isn't true when the effects are measured in an industry with only several major actors. For example you decide to test out net neutrality to see what effects it has on broadband cost. So you create a bill that says "Net Neutrality unless broadband costs go up". Then Comcast could just increase their prices to deactivate the bill. But situations like these would be easy to stop and the exception.
Re: Show HN: Guess which government programs work
#54Earlier quoted context omitted.
I hated this quiz, since it is propaganda to social science, which doesn't deserve to use the word science.
> which doesn't deserve to use the word science Why not? You don't think the outcomes of the presented solutions were evaluated using scientific method? It would be nice if you could elaborate. I understand your point, but if we were fair then software engineering shouldn't be called engineering either. Just like in social sciences, there is a lot of normative judgement and opinion compared to observation and experim…
The problem is that it is too soft. Psychology tells us that punishments work, social science tells us they have no effect? Really.
Re: Show HN: Guess which government programs work
#55I'm one of the creators of the quiz! Note that the projects were not chosen based on their results, and certainly not because the results are counterintuitive. Rather they are the ten best understood social programs we could find. You can read about the findings of our research into whether people can guess what works and what doesn't ahead of time: http://www.vox.com/2015/8/13/9148123/quiz-which-programs-wor...
I enjoyed the quiz, but I am not a big fan of the positive_effect/no_effect/negative_effect classification, because it hides the error of the measurement. Instead, I propose to use a slider, like: > "In your estimate, how does the second response program influence the rate of repeat offenses?" > Underneath, a slider on a one-dimensional axis, with appropriate labels. For example, on the leftmost side: "50% increase i…
That would be better, though quite a bit more effort!
Usually these studies only have enough resolution to say "probably worked", "no measurable effect", "probably negative", and even then are often wrong.
Re: Show HN: Guess which government programs work
#56I was thinking about why this website triggered my skeptic alarms and came up with a few things. 1. The web interface makes it hard to go back. The first thing I want to do when I see a surprising answer is reread the question. I understand this is likely just bad engineering, but it makes the whole site feel less trustworthy. 2. Some questions get summarized poorly. For example, the mindfulness question asks "What e…
That's good feedback, thank you.
Re: Show HN: Guess which government programs work
#57While the content is great, the tech seems bloody awful. Why is it so slow to load? It takes 5-10 seconds to load on both my laptop and my desktop (both It severely impacts the usability, it looks like nothing is happening and I can imagine a lot of people just close it before it loads. And it's (virtually) static content! Why doesn't it load with the opening page at least?
Sorry about that - we got a bit overloaded. It's also a beta platform, hence the loading page issue.
Re: Show HN: Guess which government programs work
#58I find it a bit unintuitive that "no change" errors are valued the same as "opposite" errors (e.g. answering "positive" when the change was actually negative). I find it far less interesting when the discrepancy is whether there is any change or not than when the effect is actually opposite to what one would apparently expect. Additionally as others have pointed out some of the questions and answers aren't very clear…
We have to just pick one outcome variable for each experiment though, or it's going to get too complicated.
Re: Show HN: Guess which government programs work
#59Earlier quoted context omitted.
> which doesn't deserve to use the word science Why not? You don't think the outcomes of the presented solutions were evaluated using scientific method? It would be nice if you could elaborate. I understand your point, but if we were fair then software engineering shouldn't be called engineering either. Just like in social sciences, there is a lot of normative judgement and opinion compared to observation and experim…
The problem is that it is too soft. Psychology tells us that punishments work, social science tells us they have no effect? Really.
Would be nice if you explained in more detail what exactly do you mean. IMHO, this is too reductionist. I can only guess:
I imagine punishment works better on children but worse on adults. I imagine it works better against psychopaths than people who have empathy. I imagine it works better on people who coldly calculate than on people who impulsively give to emotions. I imagine it works better on people from the same ingroup than from the outgroup.
Each of these items alone can explain why you can see the effect psychologically but not socially.
Re: Show HN: Guess which government programs work
#60This may be specific to me (though I'm using the latest chrome, latest mac os, so I doubt it), but the "share on facebook" link didn't work at all for me, and the facebook "like" link took me to a page to share the post on my timeline, not just "like" it. Didn't see any easy route to report such things, so here it is!