Science Fairs Aren't So Fair

"I'm not sure what's left to measure if we attempted to remove support, facilities, and motivation."

Money? Unless that's filed under support.

>it's much harder for judges to sniff out people who didn't do original work in the biological sciences category

This applies to biological research all this way up the chain, including research done at top Universities, and by "top men".

"Also a stealth observation is the hidden assumption that a negative or null result means the experiment was a failure."

I want to say that the "science" of the science fair is a sick parody of the real process of science, except the sick parody is itself now the real process.

One of the few things from grad school that still can cause me to momentarily crinkle my eyebrows in annoyance 12 years later is when I was in my machine learning class, and we were told to go use some automated decision tree learning software to go learn something from some data set. I deliberately chose a dataset and question that should have produced a null result, precisely to validate that the technique in question did in fact produce a null result. Despite the fact the professor completely agreed that I correctly converted the data, correctly formulated a hypothesis, correctly tested the hypothesis, and correctly determined the null hypothesis could not be discarded... I took a 50% deduction on the assignment. A thing that was ultimately irrelevant since I still got the max final grade in the course... what furrows my brow isn't the actual bad grade, but what it says about the mentality of what science is, even at that high level.

If that still doesn't sound scandalous to you... and it doesn't sound scandalous to a lot of people at first... consider the outcome if I had not chosen a data set that I expected to be null. Suppose I chose a data set that I expected to be able to learn something from, and I was wrong. (This happens a lot in the real world with real data, after all!) What does this mean? It means that 50% of my grade would have been dependent on me massaging the data until I did get a result. Yow! Real science should not be encouraging people to mine data until they get results like that. And one need not spend very long looking at how real science is done to see this idea is quite pervasive in "real science".

Real science is not compatible with the idea that a non-null result is "success" and a null result is "failure". Success and failure have to do with whether the result, whichever of those things it may be, correlates with reality. Ask a scientist this question directly and there's at least a decent chance they'll mouth words about how important the reality correlation is, but look to their actions and it's all "result = success".

I'm gonna be brutally honest here... my plan is that when the kids hit the science fair age, I'm going to tell them to just do what they need to do to get by, and explain to them what real science is. The two things are in direct conflict, and I'll take real science any day.

I judge high school engineering projects at the Synopsys Science Fair and California State Science Fair, and have become quite skilled in BS detection. Aside from the usual methods to detect BS, the most revealing question in my experience has been "what problems did you run into while doing this project?" I ask this question to every student regardless of whether I think their project was BS or not, and the responses from legitimate students are usually sincere admissions of incompetence, while the response from a BSer is some inconsequential roadblock that they easily overcame. Students who are willing to submit a project where they did not produce the actual content will be willing to pawn off any experience or skill as their own, and will vastly overstate their abilities. The students who are humble about what they have accomplished and understand the full implications of their work are usually the ones who produce original work and win science fairs.

This year, I saw a project where a high school senior used deep power analysis and machine learning to break AES 128 on a microcontroller (https://instagram.com/p/0JzlxBoIaJ/). The student was from a prestigious private school, and the project required an expensive high-precision oscilloscope. Naturally I assumed that he had bought the project off eBay, so I didn't hold back in my questioning. But after I began questioning him, it quickly became clear that this was just an exceptionally bright mind who was willing to put hundreds of hours into deep understanding of a topic and reading the relevant research, before embarking on some original work of his own. He iterated over many possible solutions, and had an incredible demonstration of a working one. When I asked him difficult cryptography questions, he answered them precisely. And when I questioned him on his research methodology, he was humble in acknowledging his shortcomings and truthful in citing the research that guided certain aspects of his project. It is unfair to the exceptional few students capable of producing such a project to assume that all high school students are incapable of doing graduate-level research.

I went to a national science fair with a friend who (completely on his own) came up with a novel, patentable way of reducing drag on ships. I don't remember the details, but he went to the international stage with it, justifiably, and there was a fair bit of fluid dynamics involved.

For me, science fair was the only thing that kept me going to school. (Indeed, I dropped out the next year when I was disallowed from competing.) The project that got me to the nationals was about comparing the efficacy of genetic algorithms against neural networks in a game playing task (with a multi-machine, networked simulator); looking back, it was a big mess, and there was a lot of dumbness (that, if I had had even one mentor, could have been rooted out), but it wasn't dissimilar from unambitious master's projects in the CS department at my local university. Many projects at the national level were much more sophisticated.

Like any school competitions (does the author think math and programming contests are any different?), people with ambitious parents who push their children, and especially children in private schools (that often dedicate class time to working on these competitions), have a massive presence at the higher levels. But what's nice is that you don't have to have any of that to get to those higher levels, either. Motivation is a funny thing.