Live data from Hacker News

Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

motherjones.com

91–100 of 102 posts

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#91

>Post-hoc data mining is very bad, except when you’re gunning up evidence of anthropogenic climate change, which is very good

For the record, I believe that you didn't get downvoted for suggesting that there is post-hoc data mining happening around anthropogenic climate change. Instead, it's because you're providing no evidence for this and it's in addition offtopic.

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#92

I 100% agree with the headline; especially for research papers even more than clinical trials. However, I'm a bit puzzled by the weird direction the journalist ran with this, which is straight to his preconceived notions that are not that supported by the data he's looking at. But there's a bit more to this than just that one chart. In addition to self-correction (e.g. beginning to require pre-registration of trials)…

I'll go one stronger and say "You Shouldn't Trust Any One Scientific Study You See". Individual studies can be really interesting. They're important for researchers to know about to inform their future work. But any one study - even ones that are done honestly, with good methodology and sound foundations - can be just totally wrong. There could be confounding factors you couldn't have known about that completely inva…

^ this. Especially on highly sensitive topics like medicine, I tend to be cautious when reading about the findings of any single study; whenever someone tells me "there's a study that says [flavour of the day]" my immediate reaction is to go see if there's a Cochrane meta-analysis :)

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#93

Earlier quoted context omitted.

I'll go one stronger and say "You Shouldn't Trust Any One Scientific Study You See". Individual studies can be really interesting. They're important for researchers to know about to inform their future work. But any one study - even ones that are done honestly, with good methodology and sound foundations - can be just totally wrong. There could be confounding factors you couldn't have known about that completely inva…

Yes. Science can be an adversarial process as people converge on the truth. Like a legal trial, you shouldn't take out of context any single statement from a defence or proceution lawyer as representative of the whole truth

That is a great analogy, just like scientific studies a defence or prosecution can argue to win, but in the end it's usually the evidence that swings the case...

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#94

>Post-hoc data mining is very bad, except when you’re gunning up evidence of anthropogenic climate change, which is very good

I down voted because this is (1) whataboutism, (2) I can't see the analogy.

That increased CO2 emissions should cause a temperature increase has been proposed over 100 years ago. They did not look at climate change recently and suddenly decide "lets blame CO2".

(3) the man made climate change theory, does not rest of a single study, it rests on countless ones.

(4) There is actually a scientific model explaining why we see the measurements we get. They are not digging around at random.

The case for man made climate change is pretty strong. Look at these graphs and tell me which graph correlates most strongly with global temperature increases: https://www.bloomberg.com/graphics/2015-whats-warming-the-wo...

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#95
post #4

Earlier quoted context omitted.

I don't follow this comment. What would you plot on the vertical axis?

Clickthroughs, presumably. I assume that taruz is saying that journalists should be required to say what they're investigating before publishing an investigative report. Right now, they start investigating, and if there's something outrageous (even if it wasn't what they were initially looking for), they publish. Sometimes they even skip the "start investigating" part and just put up a tip-line for anyone who has a b…

That is a natural outcome of the hyper capitalist world we currently live in. I have a mother who has been a journalist for over 40 years. I've heard for years about the pressure sell. Article titles often get changed against the will of journalists to be more click-bait like to sell more.

As people pay less for good journalism, and ad revenue is shrinking, the media is getting desperate to stay afloat.

Journalists today have significantly less independence than they had 15 years ago. Everything is far more controlled and geared towards sales.

I would like to see more alternative finance models for quality journalism.

Given how social media is actually starting to destroy democracy, it may be worth considering government grants to independent media organizations as a part of national defense. A democracy cannot function if the media is utterly broken. That means citizens are no longer capable of making informed decisions.

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#96
post #41

Earlier quoted context omitted.

Hardly equivalent "for all intents and purposes." The purposes of the layman are nothing like those of the research community.

If you put it in the public domain it is your responsibility to ensure it is digestible by the audience you publish it to. So yes... public posted information should be layman ready

Emphatically, no, it's not.

Putting something in the public domain allows other people with knowledge in the field to use the information. It doesn't mean that the lay-person should be able to digest it.

The public domain isn't some lowest common denominator clearing house of information, it's just public.

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#97
post #88

Earlier quoted context omitted.

And in fact the replication crisis in psychology suggests that the practice goes beyond just clinical trials.

The replication crisis in every field of science. Two-thirds of reported results cannot be reproduced.

If you haven't read it already, this article on the placebo effect a few days ago was interesting in how we on one hand use it as a baseline for measuring the effectiveness of drug treatments but actually can vary wildly between individuals. It's not surprising that results are so inconsistent.

https://news.ycombinator.com/item?id=18398342

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#98

Earlier quoted context omitted.

I'll go one stronger and say "You Shouldn't Trust Any One Scientific Study You See". Individual studies can be really interesting. They're important for researchers to know about to inform their future work. But any one study - even ones that are done honestly, with good methodology and sound foundations - can be just totally wrong. There could be confounding factors you couldn't have known about that completely inva…

Yes. Science can be an adversarial process as people converge on the truth. Like a legal trial, you shouldn't take out of context any single statement from a defence or proceution lawyer as representative of the whole truth

Or it’s like seeing one great play in the middle of the game. It may be a beautiful play, it might have scored the team you’re rooting for a point, but until you know the whole game you don’t know who won.

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#100
post #99

Earlier quoted context omitted.

Your remark finds difficulty in satisfying clear and present danger.

Vaccine studies

...are of a certain irreducible complexity which undergo a process of peer review prior to publication in scientific journals, and rightfully presume a minimum prerequisite background for proper interpretation? I'm sure the topic and its meta are interesting to the professional audience in which they were intended for.
Post reply on HN