How a word sparked a four-year saga of climate fact-checking
theconversation.com
How a word sparked a four-year saga of climate fact-checking
1–10 of 62 posts
Re: How a word sparked a four-year saga of climate fact-checking
#2I see a large problem in the fact that (as it seems) the original data and the computer code is not published, too. This is in my opinion a contradiction to the scientific principle that any result or conclusion has to be able to verified independently as much as possible. Though not publishing data and computer programs (and ideally a virtual machine for independent execution of the program code, too) is unluckily accepted practise in science.
If it were otherwise any critic of the result could reapply the computer program to the non-detrended data and see that the conclusion will not change. On the other hand, if the computer program and data is not published he has to trust the conclusions that the researchers came to. So the only way for an independent person to check the result is to find indirect evidence that the conclusion could be wrong. And finding out that a different data set than (wrongly) claimed was used is exactly such indirect evidence (and in my opinion nearly as far as one is able to if one doesn't have access to the used data and computer programs).
Re: How a word sparked a four-year saga of climate fact-checking
#3> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results.
> The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on 26 April, 2016, the manuscript was reviewed by seven reviewers and two editors, underwent nine rounds of revisions, and was assessed a total of 21 times – not to mention the countless rounds of internal revisions made by our research team and data contributors. One reviewer even commented that we had done “a commendable, perhaps bordering on an insane, amount of work”.
So one side you do this "mammoth" and "insane" task to review paper. On other side you block people from doing independent audit?
Climate studies decide how billions (if not trillions) of dollars are spend. People should be allowed to cross examine raw data and program source code.
Re: How a word sparked a four-year saga of climate fact-checking
#4> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results. > The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on…
If its coming with hate mail and or has a generally offensive tone, I don't think it unreasonable to be unhappy. The author doesn't appear to have blocked access. One can cross examine without being a dick.
Re: How a word sparked a four-year saga of climate fact-checking
#5I see a large problem in the fact that (as it seems) the original data and the computer code is not published, too. This is in my opinion a contradiction to the scientific principle that any result or conclusion has to be able to verified independently as much as possible. Though not publishing data and computer programs (and ideally a virtual machine for independent execution of the program code, too) is unluckily a…
While I certainly share your sentiment that both, data and analysis software, should be available, I do see a conflict of interest here that cannot be easily resolved. On the one hand, we do assert (wrongly, IMO) that competition is a necessary element to drive scientific innovation and consequently require scientific actors to adhere to economic principles, i.e. publish more and better-received papers. On the other hand, allowing researchers to keep their competitive advantage, and having exclusive access to research data certainly is, contradicts the principle of reproducability and unneccessarily restricts further research that could be done based on that data. I don't see a way to easily resolve that in the current scientific framework. One (less than optimal) approach, that is frequently taken in astronomy, is to allow exclusive access to telescope data to the members of the consortium or the principal investigators for a certain amount of time, after which it is released to public. As for the software, it's easier. if the analysis method is sufficiently well documented (and it should be in any publication), people with access to the original data can easily attempt to reproduce the results and either way they'll get a publication out of it. So there is no incentive to keep the analysis software but there is a huge incentive to keep the original data.
Re: How a word sparked a four-year saga of climate fact-checking
#6> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results. > The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on…
If its coming with hate mail and or has a generally offensive tone, I don't think it unreasonable to be unhappy. The author doesn't appear to have blocked access. One can cross examine without being a dick.
[deleted]
Re: How a word sparked a four-year saga of climate fact-checking
#7> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results. > The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on…
My understanding is that for climate science, a lot of effort is spent to debunk claims, by well-financed groups who will publish without independent peer-review but with a clear goal in mind. Allowing those groups to access the same data means that the researchers will have to spend a lot more effort explaining why those claims are unfounded. That means the researchers are not really encouraged into sharing the data.
There is a more general issue in science or any data-intensive practice (Google ranking is another example) where the argument “People should be allowed to cross examine raw data” fails to acknowledge that ‘People’ generally do not have the ability or know-how to, and those who can generally have interested financial backing. In many cases, it is positive (Open Source Map is a good example of that: my employer is interested in identifying, say, bike parking spots) but there are examples where de-anonymisation, competitive pressure, make “sharing data” a pragmatic question that needs more than principles.
Will more people who can contribute positively to climate science help if they had free access to those data? It is the case for astronomy; I am less convinced for climate science.
Re: How a word sparked a four-year saga of climate fact-checking
#8> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results. > The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on…
If its coming with hate mail and or has a generally offensive tone, I don't think it unreasonable to be unhappy. The author doesn't appear to have blocked access. One can cross examine without being a dick.
Read linked article, offensive language was used AFTER they refused to provide raw data. It took lawsuit to get them.
Re: How a word sparked a four-year saga of climate fact-checking
#9> Meanwhile, our team received a flurry of hate mail and an onslaught of time-consuming Freedom of Information requests for access to our raw data and years of our emails, in search of ammunition to undermine and discredit our team and results. > The mammoth process involved three extra rounds of peer-review and four new peer-reviewers. From the original submission on 3 November, 2011, to the paper’s re-acceptance on…
My understanding is that for climate science, a lot of effort is spent to debunk claims, by well-financed groups who will publish without independent peer-review but with a clear goal in mind. Allowing those groups to access the same data means that the researchers will have to spend a lot more effort explaining why those claims are unfounded. That means the researchers are not really encouraged into sharing the data…
Paranoid talks about "well financed groups" have no place in science.
Climate science is way more important than astronomy. It should have much higher standards for transparency, not other way around.
Re: How a word sparked a four-year saga of climate fact-checking
#10I see a large problem in the fact that (as it seems) the original data and the computer code is not published, too. This is in my opinion a contradiction to the scientific principle that any result or conclusion has to be able to verified independently as much as possible. Though not publishing data and computer programs (and ideally a virtual machine for independent execution of the program code, too) is unluckily a…
While I certainly share your sentiment that both, data and analysis software, should be available, I do see a conflict of interest here that cannot be easily resolved. On the one hand, we do assert (wrongly, IMO) that competition is a necessary element to drive scientific innovation and consequently require scientific actors to adhere to economic principles, i.e. publish more and better-received papers. On the other…
The solution should rather be that society (and publishers) does not consider papers (or more generally: texts) as research that don't also publish everything possible for independent verification (in this case the source code and raw data). If this is not the case, the text is just a plain marketing statement (in contrast to research) comparable to the marketing statements that are made everyday by lots of companies.