Live data from Hacker News

A researcher's response to people pointing out 150 errors in four of his papers

andrewgelman.com

81–90 of 105 posts

Re: A researcher's response to people pointing out 150 errors in four of his papers

#81
post #76
post #72

Earlier quoted context omitted.

>"The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive." I tried to find this a few years ago in the case of measles and it turned out to seem no such studies existed. Have you seen one?

it took a little searching, but I found one for an alternative manufacturing process of the measles vaccine. https://clinicaltrials.gov/ct2/show/NCT01536405?term=AMP&ran...

Thanks, it looks like this one was done after I was searching. This study doesn't really have the outcomes we need to measure though. There is no placebo group, they didn't check for measles diagnoses (by doctors who didn't know whether they were vaccinated), and they only measured antibody levels for 1.5 months after 1 vaccine dose in children 12-23 months old (ie before going to school when exposure usually occurs).

Unfortunately, I really don't think this study offers us anything about whether "vaccines work". You can see that the main concern is whether doctors are much more likely to diagnose measles when they do not believe a vaccination occurred, and whether introduction of the lab tests made the diagnostic definition more stringent:

“This was not a blind study, since the investigators knew which children had received measles vaccine. It seems probable that the occurrence of so much ‘measles-like’ illness in the vaccinated children was a reflexion of the difficulty in making a firm diagnosis of measles in the African child at one visit.”

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2134550/

“A likely reason for this is that the case may have been misdiagnosed as a non-specific viral illness. Measles has become relatively uncommon in Singapore with two decades of widespread measles vaccination, and especially after the second dose policy was implemented in 1998. Many primary care doctors may not even see a single case of measles in a year. This makes diagnosis more difficult.

[...]

As only approximately 7% of the clinically-diagnosed cases of measles reported locally turned out to be measles by laboratory testing, there is a need for laboratory confirmation of measles to avoid misidentification of cases and improve disease surveillance.(2)”

http://www.ncbi.nlm.nih.gov/pubmed/17609829

Re: A researcher's response to people pointing out 150 errors in four of his papers

#82

Earlier quoted context omitted.

Climate science is totally inaccessible to lay people. All we can do is trust. I'm a science teacher and when I teach climate change, it's just the bare basics and a few subjective facts. We can make cardboard greenhouses and draw pictures of IR rays getting absorbed in the atmosphere. But that's not what's in doubt. That's the stuff everyone's accepted for decades. The part people don't trust is the "it'll destroy t…

EPA has a website describing the mid and long term effects of climate change in N America. With plenty of citations a dedicated student or teacher could follow up on. Very accessible, and did not include "will destroy the world". Incidentally, your critique also applies to literally most science, including very good and obviously successful science. Can you really prove to your students that Intel doesn't have magic…

>"This figure shows projected greenhouse gas concentrations for four different emissions pathways." https://www.epa.gov/climate-change-science/future-climate-ch...

The biggest greenhouse gas by far is water vapor, not CO2. So it makes no sense to call a plot of CO2 a plot of "greenhouse gas concentrations". This type of "information" is low quality and will only make people not trust them.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#83
post #71

Earlier quoted context omitted.

Meanwhile, anti-vax is huge in liberal soccer-mom only-shops-at-whole-foods cities. The left has its fair share of science-denying morons as well.

The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive. Climate Science is a bunch of computer models are not testable. When they're run and compared to what actually happened they're always off. e.g. Name a single climate science model that predicted "the pause" starting in '98?…

> Climate Science is a bunch of computer models

All of science is eventually a bunch of models.

> models [that] are not testable. When they're run and compared to what actually happened they're always off

You do realize these statements contradict one another, right?

Also, you're wrong. There have been plenty of successful predictions: https://johncarlosbaez.wordpress.com/2013/02/05/successful-p...

> Climate Scientists don't release their model's code

False.

Some publish code before/with their papers, some don't publish at all, and some publish code with a lag after publication (to clean up the code, tussle with legal about IP and/or political issues, etc. Fair IMO as long as the code is eventually published).

The major models, including the NASA one, are open source.

Google it.

> Name a single climate science model that predicted "the pause" starting in '98?

Predicting century-long trends and predicting 5 or 10-year variations within that trend are extremely different tasks.

It's certainly possible that we have models that are exceptionally accurate on the scale of 20 or 50 or 100 years but cannot make accurate predictions over the course of 5 or 10 years.

In fact, in a stochastic system, that's exactly what you would expect to be the case -- trends that are globally but not locally discernible.

More-over, opponents to the climate change thesis don't bother to provide their own models and predictions for falsification. And when they do, they're just as wrong on small time scales and much more wrong on long time scales.

> I'm saying that labeling dissenters as deniers (reminiscent of holocaust deniers) is beyond the pale.

This largely depends on what is being denied:

  - CO2 effects climate (requires extraordinary new evidence -- we have both historical Earth data and inter-planetary data).
  - Temperatures are changing (requires new evidence -- the '98 "hiatus" as largely played out and skeptics failed to make any predictions that made it through the decade).
  - CO2 is causing temperatures to change on earth (again, requires new evidence).
  - Concrete predictions about near and mid-term effects of climate change. IMO fair game, but I'm not sure what the point is from a climate policy perspective unless we're really that selfish.
There's also an important difference between skepticism and denial.

Being skeptical of Newtonian physics and providing a competing model against which we can compare is of course fair game.

Denying Newtonian physics in favor of "no particular opinion on how the world works, and let's not try to figure out" is of course worse than useless.

In other words, make your own predictions or get off the pot.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#84
post #71

Earlier quoted context omitted.

Meanwhile, anti-vax is huge in liberal soccer-mom only-shops-at-whole-foods cities. The left has its fair share of science-denying morons as well.

The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive. Climate Science is a bunch of computer models are not testable. When they're run and compared to what actually happened they're always off. e.g. Name a single climate science model that predicted "the pause" starting in '98?…

> Name a single climate science model that predicted "the pause" starting in '98?

Wouldn't that be a failure of the model?

http://www.bbc.com/news/science-environment-38513740

Re: A researcher's response to people pointing out 150 errors in four of his papers

#85
post #53
post #12

Earlier quoted context omitted.

It depends on the situation. I work with a professor who came from industry. He once worked on a project where they did massive data collection inside the military. Over $1 million was spent over 2 weeks collecting data. A couple months after, some researcher called and said "Hey, I heard about the data you collected. Would you mind passing it on to me?". Of course the answer was no, not until we publish damn near ev…

Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one. If others use the data they should put the originators on the data as coauthors on the resulting papers.

> Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one.

I'm not saying the data should never be shared. Often times you can contact the author of a particular study and they will provide anonymized data. Most of the time people don't ask, and to take the time to prepare data for public release would offer little to no benefit.

I do believe there should be "first rights" to publish on data collected with funding from public sources. As for an exact policy by which it could be handled, I don't have one in mind. When you spend dozens of hours writing grant proposals only to have most of them turned down, it would be a sucky situation to have to make the data available for everyone before you have gotten a single paper out of it.

> If others use the data they should put the originators on the data as coauthors on the resulting papers.

It may vary by field, but generating the data alone is almost certainly not sufficient for co-authorship. [1]

[1] http://www.icmje.org/recommendations/browse/roles-and-respon...

Re: A researcher's response to people pointing out 150 errors in four of his papers

#86
post #82

Earlier quoted context omitted.

EPA has a website describing the mid and long term effects of climate change in N America. With plenty of citations a dedicated student or teacher could follow up on. Very accessible, and did not include "will destroy the world". Incidentally, your critique also applies to literally most science, including very good and obviously successful science. Can you really prove to your students that Intel doesn't have magic…

>"This figure shows projected greenhouse gas concentrations for four different emissions pathways." https://www.epa.gov/climate-change-science/future-climate-ch... The biggest greenhouse gas by far is water vapor, not CO2. So it makes no sense to call a plot of CO2 a plot of "greenhouse gas concentrations". This type of "information" is low quality and will only make people not trust them.

You're being pedantic, and worse than that, you're wrong. It's not a plot of C02.

A full description of the methodology and what exactly is being plotted is available in papers linked to here: http://tntcat.iiasa.ac.at:8787/RcpDb/dsd?Action=htmlpage&pag...

Re: A researcher's response to people pointing out 150 errors in four of his papers

#87
post #85
post #53

Earlier quoted context omitted.

Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one. If others use the data they should put the originators on the data as coauthors on the resulting papers.

> Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one. I'm not saying the data should never be shared. Often times you can contact the author of a particular study and they will provide anonymized data. Most of the time people don't ask, and to take the time to prepare data for public release would offer little to no benefit. I do believe there should b…

   generating the data alone    
Manus manum lavat.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#88
post #76
post #72

Earlier quoted context omitted.

>"The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive." I tried to find this a few years ago in the case of measles and it turned out to seem no such studies existed. Have you seen one?

it took a little searching, but I found one for an alternative manufacturing process of the measles vaccine. https://clinicaltrials.gov/ct2/show/NCT01536405?term=AMP&ran...

Here is another one, apparently we can get a drop of up to 99.5% just by switching to lab tests from clinical diagnosis: "Indeed, an average of only 100 cases of measles are confirmed annually [32], despite the fact that >20,000 tests are conducted [28], directly suggesting the low predictive value of clinical suspicion alone." http://jid.oxfordjournals.org/content/189/Supplement_1/S185....

I just don't think these alternative explanations have been investigated very well, and the blinded RCT (vs placebo) is missing. Another thing, the various proxies for "vaccine success" they use do not correlate with each other...

"Our data demonstrate that regression analysis shows only limited correlation between NT results and the ELISA values. This is in agreement with other reports [4]. Similar limitations in the correlation were also reported for other viruses like Cytomegalovirus (CMV) [10]. In case of the gamma globulin samples, the low correlation might reflect the wider spectrum and heterogeneity of the involved or measured measles antibodies." http://www.ncbi.nlm.nih.gov/pubmed/17308917

Re: A researcher's response to people pointing out 150 errors in four of his papers

#89

Earlier quoted context omitted.

My thought: ridiculous false equivalency. Conservatives don't distrust science carte blanc. Many of them will chuck money at asinine bullshit as long as it's billed as a way to kill the other team. Check out some of the more ridiculous stuff to come out of DoD. They distrust climate science. And they don't distrust the actual science, just the conclusions. And increasingly not even the most basic conclusion, just the…

Meanwhile, anti-vax is huge in liberal soccer-mom only-shops-at-whole-foods cities. The left has its fair share of science-denying morons as well.

The closest the left has to mainstream anti-vax politicians are the Green Party. They got a couple percent of the vote in the last election.

On the right, virtually all mainstream politicians deny climate change.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#90
post #30

Earlier quoted context omitted.

I hope you're being sarcastic. I'm a bit hesitant to provide examples, but a lot of top results in computer science are sold as being much more far reaching than what they really are.

Speaking from my own areas of experience, I've been struck by how much some celebrated methods and results in machine learning, broadly speaking, basically amount to tinkering without any hard generalizable proofs. In at least one case I'm aware of, there was a statistical proof later that made it clear the original researchers were close, but if they had a different dataset they would have come up with a different r…

I am very interested in the paper with the mathematical proof. Would you link it or just mention its title?
Post reply on HN