Earlier quoted context omitted.
Can you elaborate as to why? Snark is not really the goal here.
It's a perfectly valid, and indeed important question. Self-application is an important form of scientific methodology. Think e.g. of programming languages development: one of the first things you do is bootstrap a compiler.
Most Published Research Findings Are False (2005)
31–40 of 41 posts
Re: Most Published Research Findings Are False (2005)
#32Publication is a sign of persistence, plausibility and a minimal quality check. Peer review improves articles, it does not guarantee correctness.
That's also why one should be extremely careful about things that are not peer reviewed. Not because the bar is so high, but because it's so low.
Re: Most Published Research Findings Are False (2005)
#33Most code is buggy despite passing all tests. Publication is a sign of persistence, plausibility and a minimal quality check. Peer review improves articles, it does not guarantee correctness. That's also why one should be extremely careful about things that are not peer reviewed. Not because the bar is so high, but because it's so low.
Re: Most Published Research Findings Are False (2005)
#34I learned this in middle school during the Science Fair. My experiment didn't work. Up against the deadline and not wanting a 0, I simply made up the data. I got a nice big A. Probably exactly what happens in the real world too.
Re: Most Published Research Findings Are False (2005)
#35Do you think a Github for science would help?
Re: Most Published Research Findings Are False (2005)
#36Most code is buggy despite passing all tests. Publication is a sign of persistence, plausibility and a minimal quality check. Peer review improves articles, it does not guarantee correctness. That's also why one should be extremely careful about things that are not peer reviewed. Not because the bar is so high, but because it's so low.
Just because the bar is low doesn't mean its height is in any way related to its discernment of quality. Say, for example, the bar was simply based on the institution that the author is employed by. That may be a low bar in that anyone from Harvard can get a paper published, but it also doesn't actually tell us anything about the paper's validity. Whereas I wouldn't be skeptical of a good Rush University paper, simpl…
Re: Most Published Research Findings Are False (2005)
#37I learned this in middle school during the Science Fair. My experiment didn't work. Up against the deadline and not wanting a 0, I simply made up the data. I got a nice big A. Probably exactly what happens in the real world too.
Re: Most Published Research Findings Are False (2005)
#38I think the solution to this is that science publication needs to be done like science itself. Just as scientists commit to doing experiments without knowing what the results of the experiments will be, publishers need to commit to publishing experiments without knowing what the results of the experiments will be. Publishers need to be involved earlier in the process. Instead of submitting a completed research findin…
Re: Most Published Research Findings Are False (2005)
#39Earlier quoted context omitted.
It's a perfectly valid, and indeed important question. Self-application is an important form of scientific methodology. Think e.g. of programming languages development: one of the first things you do is bootstrap a compiler.
You're conflating science publishing philosophy with standard software development?
PL research is a discipline with no really feasible empirical methods for robust and replicable evaluation of programming language ideas. Using self-application is a test of internal coherence -- if your new language is not good at writing a compiler, then maybe it isn't such a good idea.
Self-application is a well-known scientific methodology. Would you trust a physical theory that predicted the impossibility of physicists?