Reminds me of an interesting Wired article from 2008: "The End of Theory: The Data Deluge Makes the Scientific Method Obsolete" http://www.wired.com/science/discoveries/magazine/16-07/pb_t...
Keeping computers from ending science's reproducibility
31–40 of 63 posts
Re: Keeping computers from ending science's reproducibility
#32Earlier quoted context omitted.
I am often told that I should keep my nose out of other science domain's business because they know more than I do. I've only heard such statements coming out of a few fields: math education, labor economics, climate science and psychometrics of race/gender. You should ignore such statements; they are nothing more than an attempt to bully you into accepting received wisdom from activists with a PhD. As an actual scie…
Education is a lot more complex than most people understand it to be. Even within the field few people actually have any understanding of how the brain and learning interact. Everything from diet, to time of instruction, to setting, to materials, to methods, to testing is all important. A classic example of which is how do you build a useful test? Well if you want to know how much learning took place you need to test…
On the contrary, I think math ed types want to keep the actual mathematicians out precisely because the mathematicians recognize how complex the field is. Every time I've been told "stay out, you don't know our field", it was because I pointed out complexity or plausible alternative explanations.
For example, a puzzle: why do SAT scores underpredict female performance in Calc 1, and overpredict grades in higher level courses? I suggested girls are more conscientious than boys, and that a study being proposed did not control for such factors. I even suggested a test of this hypothesis: compare grades in classes which are conscientiousness-weighted (25% homework, 10% attendance, tests questions are practice problems with numbers changed) to ability-weighted coursework (50% midterm, 50% final, test questions all new).
I'm not sure my idea was a good one. But rather than explain why either a) this had been tried, and didn't work, or b) why my test wouldn't work, I was simply told to leave the field to experts.
I've come up with plenty of dumb ideas in other areas, e.g., global existence of wave equations. The closest I've ever gotten to "stay out, you aren't an expert" was "Terry tried that and failed, ask him why before spending lots of time on it."
[edit: really confused why you are being downmodded.]
Re: Keeping computers from ending science's reproducibility
#33Earlier quoted context omitted.
I am often told that I should keep my nose out of other science domain's business because they know more than I do. I've only heard such statements coming out of a few fields: math education, labor economics, climate science and psychometrics of race/gender. You should ignore such statements; they are nothing more than an attempt to bully you into accepting received wisdom from activists with a PhD. As an actual scie…
Such statements also come out of biologists discussing evolution. This is not, however, evidence that they aren't really doing science. Instead it is evidence that they've been burned out explaining basics over and over again to Creationists and want to get on with their lives. However some do take the energy out for those explanations. One of the results of their energy is http://www.talkorigins.org/ . Hopefully som…
> Such statements also come out of biologists discussing evolution
This is only the case when the opposing person is pretending to be a scientist but is really a creationist. Scientists tend to make arguments based upon facts and experiments. They start to get really mad when people don't follow the rules and start bringing non-scientific "evidence" into the argument.
There is plenty of things up for debate about how evolution works, but these are the details of the theory. Due to overwhelming evidence, the debate on if is over, however the debate on how is very much alive.
Re: Keeping computers from ending science's reproducibility
#34Earlier quoted context omitted.
I am often told that I should keep my nose out of other science domain's business because they know more than I do. I've only heard such statements coming out of a few fields: math education, labor economics, climate science and psychometrics of race/gender. You should ignore such statements; they are nothing more than an attempt to bully you into accepting received wisdom from activists with a PhD. As an actual scie…
Such statements also come out of biologists discussing evolution. This is not, however, evidence that they aren't really doing science. Instead it is evidence that they've been burned out explaining basics over and over again to Creationists and want to get on with their lives. However some do take the energy out for those explanations. One of the results of their energy is http://www.talkorigins.org/ . Hopefully som…
I've actually never heard that statement coming from biologists. Usually they just say "learn more about biology, look up $SOME_SPECIES on wikipedia."
Re: Keeping computers from ending science's reproducibility
#35I am often told that I should keep my nose out of other science domain's business because they know more than I do. However, I think when they start building their science on top of computers, I start getting a say again. Here's what concerns me about this increasing use of computers: It seems like the vast bulk of these simulations are iterative, and therefore subject to mathematical chaos. How many of these researc…
In general, science hasn't been as reproducible as one would like it to be, there's so much tacit knowledge that's necessary for experiments, many unknown variables, so many demonstrations are so complicated that hardly anyone even goes through the motions. In general there are many more people writing in detail than there are people read and understand, many more people focusing on some narrow branch of their discipline than there are people with genuinely strong interest and understanding of the fundamentals.
My point, primarily, is that the danger of people using tools that they don't understand is an old one; possibly worsened by the narrow focus of modern science, and how easy computing tools have made simulations. But it's been here all along.
As far as sticking your nose into sciences, your nose is invited into any domains of knowledge of mine, but do keep your voice down: there are too many people making too many bold pronouncements without real knowledge; persistent, diligent, and quiet questioning seem to be much more effective at getting the heart of the matter.
Re: Keeping computers from ending science's reproducibility
#36Earlier quoted context omitted.
Education is a lot more complex than most people understand it to be. Even within the field few people actually have any understanding of how the brain and learning interact. Everything from diet, to time of instruction, to setting, to materials, to methods, to testing is all important. A classic example of which is how do you build a useful test? Well if you want to know how much learning took place you need to test…
On the contrary, I think math ed types want to keep the actual mathematicians out precisely because the mathematicians recognize how complex the field is. Every time I've been told "stay out, you don't know our field", it was because I pointed out complexity or plausible alternative explanations. For example, a puzzle: why do SAT scores underpredict female performance in Calc 1, and overpredict grades in higher level…
Honestly, SAT scores vs. Performance is the type of (mostly bad) research I am talking about. The SAT is not designed to test math ability. It's focus is on how likely a student is to finish their freshman year of college and it does that fairly well. You can do a lot of useless research in this area and it tells you next to nothing. If you want to predict a highschool students ability in advanced collage math classes you could design a test that did that. But, people have easy access to SAT data so that's what they look at.
If you want to see real and useful research look into how long the optimal study period. There are significant and useful study’s which suggest 2 hours of nonstop instruction is less useful than two one hour periods with a moderate break in-between. Yet how many collage lectures follow this approach.
Re: Keeping computers from ending science's reproducibility
#37Speaking of reproducibility, my friend getting a PHD in finance told me he was writing a paper using the data from some brokerage. I asked if he would publish the data and he told me it's confidential. I talked myself blue in the face trying to explain how science doesn't work if you don't give people enough information to reproduce your research! I couldn't get him to understand though. Arggh so frustrating.
Assuming the data was some sort of financial time series then that means lots of people already have the data (or something close enough to it). So even though he can't publish the data he can say what the data is and when and how it was collected and that should be enough for most people in the field reproduce the results.
Re: Keeping computers from ending science's reproducibility
#38I am often told that I should keep my nose out of other science domain's business because they know more than I do. However, I think when they start building their science on top of computers, I start getting a say again. Here's what concerns me about this increasing use of computers: It seems like the vast bulk of these simulations are iterative, and therefore subject to mathematical chaos. How many of these researc…
I am often told that I should keep my nose out of other science domain's business because they know more than I do. I've only heard such statements coming out of a few fields: math education, labor economics, climate science and psychometrics of race/gender. You should ignore such statements; they are nothing more than an attempt to bully you into accepting received wisdom from activists with a PhD. As an actual scie…
I agree with the sentiment you express and thank you for pointing it out.
I do think that thinking outsiders should stay out is uncommon in the field of math though. I am currently a math grad student, and while I am only a grad student what I have seen of the mathematical community makes it seem like it is very open to cross fertilization from other sciences and from amateurs.
Admittedly he is historical, but Fermat (professional judge when he did most of his mathematical work) is revered and there are some more recent amateurs that have made genuine contributions. Much of mathematics is inspired by other fields such as physics.
I admit I know some professional mathematicians were tired of finding flaws in amateurs proofs of Fermat's Last Theorem and rebuting supposed refutations of Godel's diagonal proof that the reals are uncountable, but it is very different to be tired of dealing with a constant stream like that than to be hostile towards outsiders in general.
From my limited perspective, I think math is one of the fields most open to outsiders and amateurs coming in and making real contributions, both culturally and in the fact that we rarely need expensive experimental setups so the barrier of entry (in terms of money and equipment) is low.
Re: Keeping computers from ending science's reproducibility
#39Earlier quoted context omitted.
I am often told that I should keep my nose out of other science domain's business because they know more than I do. I've only heard such statements coming out of a few fields: math education, labor economics, climate science and psychometrics of race/gender. You should ignore such statements; they are nothing more than an attempt to bully you into accepting received wisdom from activists with a PhD. As an actual scie…
Education is a lot more complex than most people understand it to be. Even within the field few people actually have any understanding of how the brain and learning interact. Everything from diet, to time of instruction, to setting, to materials, to methods, to testing is all important. A classic example of which is how do you build a useful test? Well if you want to know how much learning took place you need to test…
PS: Education might seem like a flaky field and much of the discussions are devoid of good science. At the same time there is a lot of great research that has been which has real and important implications.
My wife and I were discussing this recently. She is in an unusual program where she should graduate with both a masters in history and an education certificate with hopes to teach in history at the High School level after that.
This is both second hand an anecodatal, but from what she tells me the education courses she takes are less rigorous and generally easier than the history courses. But the reason for this is that the education course she is taking are vocational. They are trying to teach her how to get teaching done in a modern classroom, not the science behind it.
It is akin to learning to the difference between training to be a mechanic and a mechanical engineer. Both are very hard and important jobs worthy of respect, but they have very different focuses. The education courses are focused on learning the technique of teaching effective, it looks like most of the research on the theory of effective education is done in departments like psychology, sociology, etc.
Re: Keeping computers from ending science's reproducibility
#40Earlier quoted context omitted.
On the contrary, I think math ed types want to keep the actual mathematicians out precisely because the mathematicians recognize how complex the field is. Every time I've been told "stay out, you don't know our field", it was because I pointed out complexity or plausible alternative explanations. For example, a puzzle: why do SAT scores underpredict female performance in Calc 1, and overpredict grades in higher level…
Honestly, SAT scores vs. Performance is the type of (mostly bad) research I am talking about. The SAT is not designed to test math ability. It's focus is on how likely a student is to finish their freshman year of college and it does that fairly well. You can do a lot of useless research in this area and it tells you next to nothing. If you want to predict a highschool students ability in advanced collage math classe…
What makes SAT vs performance bad research? The SAT may be worse than some specialized test, but so what? Breast self exams suck in comparison to mammograms, but that doesn't make studies into breast self exams bad research.
The methodology is what makes research good or bad.