Gates Foundation study: We’ve figured out what makes a good teacher

Actually they have figured out how to evaluate teachers based on statistical models based on their repeatability of success. Finding a 'great' teacher is much more than finding the top rated teacher in my opinion. What qualities makes such a teacher good is a totally different matter. The latter was what I was hoping to find.

I can't say whether the findings are correct or not, but I like this:

>> The teachers who seemed to be effective were, in fact, able to repeat those successes with different students in different years, the researchers found. Their students not only scored well on standardized exams but also were able to handle more complicated tests of their conceptual math knowledge and reading and writing abilities.

To everyone who keeps saying "standardized tests are bad", please cut it out. A particular code benchmark may be useless, but benchmarking overall is good. Existing standardized tests may be bad, but in general, having some way to make objective measurements and adjust accordingly is the basis of all human improvement.

And yes, we should reward good performance, if we can determine what good performance is. How else do you expect to get more of it? It is incredibly difficult to make incentives that actually do what you want rather than encourage gaming of the system, but that doesn't mean we shouldn't try.

In short, let's stop debating whether we should measure and incentivize, and start debating how.

I'm not sure that the argument is even that existing standardized tests are bad so much as that they are given too much priority. They should not be the only or primary means by which teachers, students, districts, etc are benchmarked (and in turn, rewarded or disciplined). "Teaching the test" is the problem, not necessarily the tests themselves.

I grew up in the UK in a school where the emphasis of our entire education was "the test." Teachers entirely forgot how to teach a subject and our whole time was taken up with what essentially amounted to "test prep."

I did pass their tests, but did I get a good education? Frankly, no, I don't think I did. I mean statistically I got a great education, but it seems we have forgotten why we even educate to begin with...

Do we educate so people look smart on some random league table? Or do we educate so people are actually more well rounded, have a better grasp of the world around them, and do better as adults?

I'll have to look at the prior reports, but this 3rd report is somewhat disappointing in terms of its statistical rigor.

They do provide weightings based on optimal predictions for maximum accuracy with predictions for state tests scores, although those weights appear to assume a simple linear correlation. Even then, the data suggests that in class observations are almost a complete waste of time (single digit weightings), and that student surveys decline in value as the kids mature (no shocker: with younger kids, how much they like the teacher has a much more profound impact on their success at learning). In all cases the kids seem to be a more reliable indicator than the observations of professionals! This may just be an indication that assessments based on more frequent observations are more accurate, but the only way we can really get a sense of that is if they also had peer observations from team teaching environments... which are completely absent.

That's not the worst of it though. The worst is the rest of the data. It is based on some fairly arbitrary presupposed models for combining these different assessments. No breakdown of the individual components of these assessments (other than "English" & "Math") and no attempt to discern a model from first principles.

There are some other factors as well. Aside from some outliers, the accuracy of predictions seems kind of suspect. Most of the outcomes are within +/-0.05 standard deviations, and a majority of them are within what looks like +/-0.025 standard deviations. These are pretty minuscule variances (0.25 was supposed to represent a year of schooling, so 0.025 represents maybe 15 school days worth of progress at best), and it is not hard to see that without some of those outliers, the regression line doesn't look so pretty (particularly for English). I also didn't see any metrics on the variance of the data within a classroom, which I suspect makes any predictive value look limited at best.

Ugh. I really hoped that Gates would make sure there was more statistical rigor with an analysis like this. :-( I hate to say it, but the conclusions of the study are not well supported by at least what is in the final research report.