> “The report concluded that there were observable, repeatable and verifiable ways of measuring teacher effectiveness,” wrote Gates in the letter. Anonymous student surveys that asked such questions as “Does your teacher use class time well, get class organized quickly, help you when you are confused" – were proven to provide useful feedback as were reports from trained professionals observing teachers at work.
Students are notoriusly bad at rating their professors. This was proved with almost ideal control groups at the Air Force Academy [1] and again with groups of trained professionals/graduate students who learned first hand about the 'Dr. Fox Effect' [2]. Even teachers don't seem to like the teacher evaluations done by students [3].
Anecdotally, I can say that most students in my college classes either didn't show up on the survey days, or they walked out the door as soon as the surveys were being handed out. There's also no incentive to provide useful feedback from the student's perspective. If you're taking a survey about the class, it means the class is over and you'll probably never see that professor again, so why bother? I made an effort only because I felt an obligation to help future students, but I'm not sure there are many kids in college who share that feeling.
> Mary Ann Stavney, a high school “Master Teacher” profiled in the annual letter, spends 70% of her time observing other teachers, meeting with them and providing input. The problem, of course, is that this kind of measuring, particularly the hands-on observation in classrooms, is costly, adding about 2% onto payroll.
So you can have cheap and unreliable measurements, or you can have accurate but costly ones. Who's going to pay for the latter? The rating agencies? The schools? The students? Imagine the costs to enact such a program across all colleges in the U.S. alone -- some of the larger state schools easily have > 1,000 teaching faculty across a myriad of disciplines, and they're teaching increasingly diverse student bodies.
The thought of trying to implement a thorough, standardized program of that scale is mind boggling. And that's probably why we've been facing this dilemma of measuring teacher effectiveness since the day the first schools opened. Bill Gates is right that we have a serious problem, but it doesn't sound like he's any closer to a solution.
[1] http://voices.washingtonpost.com/college-inc/2010/06/study_h...
[2] http://en.wikipedia.org/wiki/Dr._Fox_effect
[3] http://en.wikipedia.org/wiki/Course_evaluation#Criticism_of_...