Kids should be building rockets and robots, not taking standardized tests

On the larger point, testing learning is hard in principle in that you are usually testing theory. Practice is the basis of theory and much harder to test.

But even then, writing a good test to test theory is very hard. Most objective test questions on university tests are terrible and only slightly, if at all, more likely to reward the testee for knowing the theory.

Programming a pong game would probably seem like genius level intelligence to many outsiders. Sure, most here don't think that, right? That doesn't mean you understand serious computer concepts and the sort of things that are Knuth's "Art," right?

It's fun to say "Who cares about solving these dumb, boring algebra problems, I can build a rocket by researching most of the things on the Internet." Sure someone who builds these kinds of things is probably more intelligent or perhaps had the chance to be more technical than your average everyday American but when you get into the details, I think we're talking about people that will successful in STEM and I just don't take seriously this romanticism of "project based learning."

I mean, just because you can follow directions and build a rocket, doesn't mean you're going to be fit to work as a Quant or something like that. This just seems obvious to me.

Of course, it goes without saying that you probably should have these kinds of projects. Definitely. These are great experiences. I doubt computer science would have even existed if people just had the attitude that "book learning" was for the birds or whatever and just "built stuff."

When I was about 12 or 13 I dropped out of school.. I was too bored, too frustrated, and too unhappy to actually stomach it any longer: the unsightly pleasure adults took in dominating children, the teachers who knew less than I did but refused to admit it, the way smart kids often ended up at the bottom of the pecking order... The only thing I enjoyed was writing simple QBasic games for my friends and me to play.

After a protracted fight with my parents (I actually went on strike on the roof of our house) I got what I wanted and they let me drop out of school. This was probably the best single decision of my life, and the credit really goes to them for being brave enough to allow it.

I started out teaching myself C++ templates, and the basics of electronics. I got obsessed with high voltage, and started dumpster diving to find good parts. Once, after I took a (harmless) 40kV shock, my mom started to worry she'd come back from her teaching job to find me dead on the floor. I convinced her I knew the actual parameters of heart stoppage well enough to avoid this!

I built plasma globes and an improvised vacuum pump. I started rewiring our telephone lines. I made an amplifier. I built a coil gun and tuned it using my computer's sound card as a digital storage oscilloscope. I taught myself Python, and dived into the theory of digital signal processing.

I even had the good fortune of acquiring some Perl contract work through a family friend. Using the money to buy my own computer at 14 was probably the most satisfying experience of my life. Old school-friends would come to visit and oggle my enormous 21" CRT. That made me realize that I didn't have to follow the ordinary rules to survive in this world.

All the while I took distance education courses, a measure my parents insisted on (luckily). When I got high grades on A-level physics, I got the opportunity to attend university. Still a year young, I ended up studying pure math at the best math university on my continent (lots of Soviet bloc math emigres who moved to Africa for the sun).

The weirdest contradiction for me was hanging out with many of the other "nerdy kids" in math and physics who had gone the traditional route, and finding out that they really weren't all that interested in what they were doing. They seemed to be more consumed with perpetuating their high marks. Most frustrating of all was that achieving those high marks often boiled down to the most 'mercenary' kind of pillaging of old exam papers -- actual understanding didn't have much bearing on it.

Pure math was different -- you really had to think, be creative, and understand things to achieve good grades, at least in the more advanced courses. And sure enough, that was the one discipline I got very good grades in. But funnily enough, only when it got hard. For the first 2 years I didn't realize I had any interest in math at all.

By and large, my experience of formal education is best summed up by one little anecdote: I once handed in a 3rd-year physics practical in which I had taken an extraordinary large amount of care: I did the experiment several times because to better resolution on the spectrometer, I researched error propagation and did a more principled error analysis, I wrote it up carefully in LaTeX, and so on.

I got a mediocre grade for the practical. In fact, my explanation of how I had used a genetic algorithm to fit Gaussians to the spectrographic peaks had the word 'genetic' crossed out and replaced with 'generic' -- the ignorant tutor thought I had just misspelled it!

In contrast, a classmate put some last-minute work together that had (illegitimately) small error bars, and got 90%. At that point I lost interest in getting good grades in that subject, and stopped trying.

There is a happy ending: industry proved to be very meritocratic indeed. And my current job means I can actually bring all this random knowledge I have to bear, and bootstrap my way into new fields (like, recently, graphical models).

What are my conclusion here? I think the main conclusions are:

* Teaching yourself on your own has the benefit that you get used to doing what you do because you love it, not because people reward you for it. And this is what really matters in the end.

* Metrics to reward behavior will almost always tend to corrupt the behavior they are meant to be promoting (this has been enshrined as http://en.wikipedia.org/wiki/Campbells_law). Grades are the best example of this.

That's not the point of hacking.

If I were to get some students to play with robotic kits, I'd give them the parts, instructions and let them do whatever they want. If you remove the "discovery" part, you get something not much different than standarized test itself. If you have kids follow instructions precisely, well, they may as well just watch how it's done on youtube. They won't learn much more from it anyway.

Adding to other excellent answers, there's also this thing that children are not robots. You can't just open a channel and upload the knowledge to their brains. You need to work them on emotional level, make them curious / interested in what they are learning, even if because it boosts efficiency tremendously.

> It would be nice if the children were engaged, and if the teacher had a clever way to even make it "fun" at times, great. But it's not a prerequisite.

Yes, it is. Even teachers who are experts in their domains may (and in academia, often are) poor teachers. Because it doesn't matter what you say (they can read up the details in textbook/on Wikipedia; that's what they're there for anyway). It matters how you say it.