Peer Review is No Panacea

Post by jerf » Fri, Jul 09, 2010, 1:13 AM UTC

thaumaturgy wrote: post #12

jerf wrote: post #7

Earlier quoted context omitted.

I fail to see your point. The core claim that there are people who put far too much stock in the words-treated-as-magic "peer review" is definitely true; I encounter them all the time, to the point that I've also come to recognize it as a pattern. The fact that this is not true is, as you say, tautological. But some people need this fact brought to their attention, as it is clearly not generally understood. I've come…

Nicely said. For me, there are two caveats to this: 1) the mere possibility that a discipline of science might be wrong does not mean it is wrong, and unfortunately the possibility of wrongness is often used as an argument where it shouldn't be; 2) the various sciences tend to be wrong in a very specific way, and for some reason this often gets ignored in discussions like this. The last time I can think of the popula…

Reversals occur too. Tectonic plates, stress causes ulcers, neural networks are the one true future of AI, and those are just three off the top of my head. Rare for an entire science to be rewritten, sure, but trying to contain it by saying it's always just refinements betrays an unfamiliarity with scientific history too.

Post by thaumaturgy » Fri, Jul 09, 2010, 4:23 AM UTC

dantheman wrote: post #20

thaumaturgy wrote: post #16

Earlier quoted context omitted.

I wasn't being hostile (this time), I was just having a little fun. What you're grappling with here is basically the "I don't know what I don't know" problem. Other people on HN have talked about this before, but basically, there's the knowledge you know, the knowledge you know you don't know, and the knowledge that you don't know you don't know. How can you possibly begin to challenge an assertion in a scientific pa…

I didn't think you were being hostile, and I hope you don't detect any hostility from me, none is intended. There are many incentives to overstate the validness of ones claims and unfortunately it weakens the overall strength of the field. I'm not asking that they make it understandable to the layman, but perhaps on any recommendation they could put confidence intervals on predictions. Here's a presentation by Carl W…

I apologize for taking so long to respond -- I needed to get to a point where I had the time to properly listen to and consider the video. (So I waited until it was time to shave. Laptops are wonderful. ;-)

I went into the video with an open but critical mind, and attempted to stay that way even as I realized that he was presenting a point of view which I would not usually agree with. So, while I think he presented some casually interesting points, I also find it curious that he seemed to be committing some of the very same errors which you are criticizing others in the scientific community for committing, as well as other errors which even I -- as a layman -- was able to catch.

It's entirely possible that his presentation is merely incomplete, and that he has good reasons for drawing the conclusions that he was presenting, and he merely couldn't present those reasons to a non-scientific audience. (Which would be pretty much my side of the argument in this thread, so I'm totally willing to accept that possibility.)

I won't devote the time to a point-by-point analysis of his presentation, unless you express enough interest in continuing this to make it worthwhile, but in short:

+ He completely -- and carefully -- ignores various geological records which go back many hundreds of thousands of years, and which have a high confidence of accuracy in the climatological community;

+ Those records do present a regular semi-chaotic rhythm reminiscent of a strange attractor, and our era appears to currently be the zenith of that rhythm;

+ And if those records are accurate, then current measurements exceed any previous maxima of the last many hundreds of thousands of years;

+ He seems for some reason to draw the analogy that the climate is correctly modeled as a chaotic system of some sort -- i.e., one which is extremely sensitive to initial conditions -- even though this is still a matter of debate within the climatological community;

+ While the Titanic may arguably be one example where taking no action may have resulted in a more favorable outcome, coming up with or inventing countless counter-examples is trivial, so this does not support his case at all;

+ His "gambling models" proposal is quite silly, if for no other reason than that one of the biggest unpredictable parts of the climate are related to human activity.

A lot of the rest of his presentation seems to be an argument-from-ignorance. I think he is framing the discussion in such a way as to suggest we have less information and less knowledge than we actually do, and then within that frame, he is arguing that we haven't the knowledge to decide what to do.

But I'll think about it some more.

EDIT: Funny, I pulled up a random-ish TED video (searched for "climate"), and ended up watching a 4-minute presentation which is pretty much exactly the point I've been trying to make: http://www.ted.com/talks/lang/eng/rachel_pike_the_science_be...

Re: Peer Review is No Panacea

Re: Peer Review is No Panacea