How Wikipedia influences judicial behavior • Hacker News Board

Re: How Wikipedia influences judicial behavior

#71

Post by asdff » Thu, Jul 28, 2022, 5:19 AM UTC

pdonis wrote: post #70

Spivak wrote: post #61

Earlier quoted context omitted.

https://nulib.github.io/moderndive_book/7-causality.html https://bolt.mph.ufl.edu/6050-6052/unit-2/causation-and-expe... https://towardsdatascience.com/establishing-causality-part-1... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6235704/ https://escholarship.org/uc/item/42v4w8k1 http://ippsr.msu.edu/public-policy/michigan-wonk-blog/random... https://www.cs.cornell.edu/courses/cs1380/2018sp/textbook/ch... I’ll leave…

> RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. No, they're not. The real "gold standard" in science--the standard that prevails in, for example, physics or chemistry--is a controlled experiment . Not just a "randomized controlled trial", but a controlled experiment , where you can actually dictate exactly what state the things you are going to experiment on start out in…

They did do a controlled experiment here. They had articles in a treatment group and articles in a controlled group. What are the shortcomings you have in mind when you say that "controlled experiments simply cannot be done" when it appears that they have done a controlled experiment?

Re: How Wikipedia influences judicial behavior

#72

Post by asdff » Thu, Jul 28, 2022, 5:23 AM UTC

bjornsing wrote: post #67

asdff wrote: post #62

Earlier quoted context omitted.

What they did was fine with this setup. You might enjoy reading this article that covers the very perceived issue you are having with the research: https://towardsdatascience.com/establishing-causality-part-1...

As I understand it a randomized controlled trial (RCT) can determine causation. But I don’t think “what they did with this setup” was an RCT. A real RCT in this context should IMHO assign new cases randomly to a “treatment” or “control” group. In this setup they assign legal precedents randomly, but all new cases are effectively under “treatment”. That’s not the same thing.

How are new cases effectively under treatment? They identified 150 decisions, wrote articles for each, and assigned half randomly to treatment or control. they did not consider newer cases beyond this set of 150. An RCT is a type of randomized experiment, but all determine causation:

https://en.wikipedia.org/wiki/Randomized_experiment

Re: How Wikipedia influences judicial behavior

#73

Post by jacobolus » Thu, Jul 28, 2022, 5:25 AM UTC

setr wrote: post #68

sien wrote: post #38

Earlier quoted context omitted.

Not always. Any particular two or more wikipedians who decide they don't want something in can sometimes stop it. For example, in Australia there is a body that does sport participation statistics, Ausplay. They do this every year. It's a great source for sport statistics on Australia. Two wikipedians decided that these statistics were not permissible in the sport in Australia article. They won : https://en.wikipedia…

If I’m reading this right, I’m seeing a vote taking place in the “Survey” section, with 5 votes invoked — 2 no, 3 yes. And an end result of No Consensus. Framing this as if 2 Wikipedians exercising outsized power to produce this ruling seems disingenuous at best. And their basic objection (I only bothered skimming) of bias and ambiguity in the source/data/methodology seems fairly reasonable on its face; whether it’s…

sien: If you think this decision was wrong you can try to bring it up again, and point out the discussion in Wikiprojects or other community pages where a larger number of Wikipedia editors will see it. If you could only get 3–4 people interested enough to discuss and nobody could come up with an acceptable compromise, that’s not really the fault of the process. If you think these particular editors did something improper, you can escalate to further community processes designed for dealing with various abuses.

The nature of any large project is that people will disagree and not everyone will get precisely what they want.

Re: How Wikipedia influences judicial behavior

#74

Post by dukeofdoom » Thu, Jul 28, 2022, 5:27 AM UTC

No post body was provided.

Re: How Wikipedia influences judicial behavior

#75

Post by DiggyJohnson » Thu, Jul 28, 2022, 5:27 AM UTC

asdff wrote: post #71

pdonis wrote: post #70

Earlier quoted context omitted.

> RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. No, they're not. The real "gold standard" in science--the standard that prevails in, for example, physics or chemistry--is a controlled experiment . Not just a "randomized controlled trial", but a controlled experiment , where you can actually dictate exactly what state the things you are going to experiment on start out in…

They did do a controlled experiment here. They had articles in a treatment group and articles in a controlled group. What are the shortcomings you have in mind when you say that "controlled experiments simply cannot be done" when it appears that they have done a controlled experiment?

Because the experiment itself isn’t closed.

Re: How Wikipedia influences judicial behavior

#76

Post by asdff » Thu, Jul 28, 2022, 5:37 AM UTC

DiggyJohnson wrote: post #75

asdff wrote: post #71

Earlier quoted context omitted.

They did do a controlled experiment here. They had articles in a treatment group and articles in a controlled group. What are the shortcomings you have in mind when you say that "controlled experiments simply cannot be done" when it appears that they have done a controlled experiment?

Because the experiment itself isn’t closed.

What do you mean by closed?

Re: How Wikipedia influences judicial behavior

#77

Post by pdonis » Thu, Jul 28, 2022, 5:48 AM UTC

asdff wrote: post #71

pdonis wrote: post #70

Earlier quoted context omitted.

> RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. No, they're not. The real "gold standard" in science--the standard that prevails in, for example, physics or chemistry--is a controlled experiment . Not just a "randomized controlled trial", but a controlled experiment , where you can actually dictate exactly what state the things you are going to experiment on start out in…

They did do a controlled experiment here. They had articles in a treatment group and articles in a controlled group. What are the shortcomings you have in mind when you say that "controlled experiments simply cannot be done" when it appears that they have done a controlled experiment?

> They did do a controlled experiment here.

No, they didn't. You can't do a controlled experiment on humans. Nobody has a "human source" that can stamp out a series of humans that are identical in all respects, to be used in an experiment, the way physicists have "particle sources" that can stamp out a series of identical particles. That's what "controlled experiment" means. The fact that they call one group a "control group" does not mean it's a controlled experiment. Humans can't be controlled to the degree required.

Re: How Wikipedia influences judicial behavior

#78

Post by pdonis » Thu, Jul 28, 2022, 5:51 AM UTC

asdff wrote: post #76

DiggyJohnson wrote: post #75

Earlier quoted context omitted.

Because the experiment itself isn’t closed.

What do you mean by closed?

I think he means that you can't isolate and control the environment of humans the way that physicists or chemists can isolate and control the environment of particles or molecules that they are experimenting on. That's an additional issue to the one I raised in another response just now in this subthread.

Re: How Wikipedia influences judicial behavior

#79

Post by AB1908 » Thu, Jul 28, 2022, 6:22 AM UTC

dhosek wrote: post #35

jacobolus wrote: post #16

Earlier quoted context omitted.

It is entirely possible to fix those mistakes (one at a time) on Wikipedia.... if you do the research work to find out what happened. But this takes significant effort (like, a half-day of research to sort out one claim), and then sometimes back and forth with other Wikipedians to convince people that you actually chased down the real story. The problem is that for every mistake someone is willing to put effort into…

I wrote about some interesting discrepancies within Wikipedia for my sporadic newsletter last year: https://dahosek.substack.com/p/something-interesting-eda

This was such a fun article. Please write more!

Re: How Wikipedia influences judicial behavior

#80

Post by asdff » Thu, Jul 28, 2022, 6:25 AM UTC

pdonis wrote: post #78

asdff wrote: post #76

Earlier quoted context omitted.

What do you mean by closed?

I think he means that you can't isolate and control the environment of humans the way that physicists or chemists can isolate and control the environment of particles or molecules that they are experimenting on. That's an additional issue to the one I raised in another response just now in this subthread.

You can when you have many samples divided into treatment and control groups that are otherwise identical, that overcomes any biases or confounding variables that might be influencing the design and ensures that what you are seeing is causal.

How much are physicist or chemists really controlling in the lab setting? There could be plenty of confounding variables in their experiments too. Maybe "RT" in this lab for that publication for that experiment is actually 75*F and its 71*F in your lab, or you are at different elevations. Maybe no one calibrated the instruments for years. Maybe the reagent wasn't fresh and absorbed too much moisture or oxygen from the room. Maybe an undergrad dropped the balance on the floor and was afraid to tell anyone.

To overcome those potential confounding variables and other biases, chemists and physicists often turn to the exact same statistical tests being employed by people in the social sciences. Technical replicates are the norm in hard scientific experimental design because of how many biases could be present in the laboratory. It's a chaotic environment. Good experimental design builds robustness no matter what your topic is.