So if the teacher evaluation method is alleged to be inferior, please provide a new evaluation method which is superior to it.
Can't? Won't? Then be quiet.
The 50 million dollar lie
31–40 of 159 posts
Re: The 50 million dollar lie
#32The author's argument for this post is rather poor. He states that the raw data plot is noisy and weakly correlated, thus no relationship can be drawn by averaging the data into sets of buckets.
While the author is correct in that averaging hides the variance of data, he does not try to answer the question of "Can a trend be found from noisy data?" or show other accepted statistical methods that show no correlation in the data.
Instead the author dismisses the report from Gates through name calling: "It seems like the point of this ‘research’ is to simply ‘prove’ that Gates was right about what he expected to be true. "
Re: The 50 million dollar lie
#33I suspect there'll be a lot of differing opinions on this, and I'm looking forward to seeing the discussion. What would be helpful would be if people could say how much experience they have in hard statistics, and how much what they say is driven a priori from the data. I know that hackers, in particular, have real problems with "Argument from Authority", but stats is one place where it's really, really easy to go wr…
Do you have a convenient link to the data set you used to generate the first plot in your article?
I think you're making a mistake that I see over and over again here on HN. Not my data, not my article. I just submitted it here.
Re: The 50 million dollar lie
#34To build intuition, let's consider a hypothetical society. We divide a group of people in two. The first group rolls a die with 100 sides labeled from 1 to 100 in even increments. The second group rolls a die with labeled 1 to 110 in even increments. The roll determines their annual ability level which is unknowable. If you were hiring, which group would you prefer? Is it fair to the second group?
Note that this isn't strictly equivalent for several reasons. The game above the error is part of the generating process instead of the measurement process. Also the distribution is district rather than continuous. The key commonality is that a weak predictor at an individual level can be a meaningful predictor at a group level.
Re: The 50 million dollar lie
#35I know nothing about the domain of teacher measurement, and have no opinion on it. But the first chart in the blog post, with the big mass of blue on it, is surely not the strong evidence of weak correlation, that the author is making it out to be? There could still be a strong correlation in that data, even if it looks like a blob of blue - because there are so many data points on that chart, that we can no longer t…
It seems to me that the evidence of weak correlation isn't quite as weak as you imply :-) If there's a dense line in there, how come it's not sticking out of the blob even a little? Do actual phenomena often yield "deceptive" blobby scatterplots with dense lines hidden inside them? That seems apriori improbable to me, thus increasing the Bayesian probability that the blobby scatterplot is not a misleading picture aft…
>If there's a dense line in there, how come it's not sticking out of the blob even a little?
It is. If you look at the density in the upper right of the cloud, it is clearly clustered around the x=y line. The correlation may not be very strong, but without extra information about how strong of an effect would constitute clear evidence in favor of including the value add metric, we can't say anything more about it.
> Do actual phenomena often yield "deceptive" blobby scatterplots with dense lines hidden inside them?
Absolutely. You're looking at a misapplication of a plotting technique - a scatterplot with substantial overplotting. The author may be convinced that there's no correlation there, but by presenting this plot they provide no evidence in support of their point.
Re: The 50 million dollar lie
#36I know nothing about the domain of teacher measurement, and have no opinion on it. But the first chart in the blog post, with the big mass of blue on it, is surely not the strong evidence of weak correlation, that the author is making it out to be? There could still be a strong correlation in that data, even if it looks like a blob of blue - because there are so many data points on that chart, that we can no longer t…
A heatmap would be more revealing, but I'd eat my hat if it weren't a standard distribution (and I had a hat). It's nature (but hey, anything's possible). The scales are only slightly skewed, but we're not talking logarithms. The merit of the scales ("value-added") on the other hand, is very questionable.
But the relation, whatever it is describing, is pretty weak since the ellipse isn't very irregular (it's only "slightly" irregular). http://www.emeraldinsight.com/content_images/fig/08702801090... Think Kepler's laws.
Edit: The first post in the series shows a scatter of the direct (non-delta) scores, and it looks like San Fransisco in the morning: http://garyrubinstein.teachforus.org/2012/02/26/analyzing-re...
Re: The 50 million dollar lie
#37So if the teacher evaluation method is alleged to be inferior, please provide a new evaluation method which is superior to it. Can't? Won't? Then be quiet.
The idea would be not to waste numerous resources and money implementing a broken idea and spend some time coming up with a better one. Why put the education of our children and the careers of good teachers on the line with such a poorly thought out and half-assed idea?
Re: The 50 million dollar lie
#38There is no objective way to measure teacher performance. Any evaluation method that can be written as a list of rules can and will quickly be gamed.
The thing is, it's easy to find out who the best teachers are. Simply ask the students, parents, and staff. They all know who the good ones and the bad ones are - and that can't be gamed.
Re: The 50 million dollar lie
#39This is a surprising blog post in that I draw the complete opposite conclusion the author does. The author seems to think that the averaging hides volatility (which it does), which leads to incorrect conclusions drawn. Whereas to me it looks like it removes visual noise to show an actual trend. In his original scatter plot, because of the big ball in the middle it's easy to handwave and say, "look a random blob" -- b…
You're missing the point. The author agreed that there is a slight positive correlation in the data. But the averaging makes it look like there is a strong positive correlation in the data, which there isn't. In other words: without the averaging, you are seeing the full implications of the data: slight positive correlation, but a lot of scatter--i.e., not much meat there. With the averaging, you are seeing only the…
Huh? The averaging doesn't give you only the positive correlation. It simply averages it. If the correlation was negative, you'd see that just as plainly. Averaging doesn't change the trend -- it just removes the volatility from the graph and makes the graph more readable (what's the density at y=x? I suspect it's a lot denser than he'd have you believe, but you can't tell from the chart at all).
Sure, you could add a volatility measure as well, but the trend is clear -- you'd rather have your child by a teacher with a high teacher score than a low one. Or are you saying that over 12 years of schooling you don't think you'd have any preference having your child taught by teachers with a score in the lowest 5%ile versus a teacher with a predicted score in the top 5%ile? Of course there's some volatility, but over time, it's seems pretty obvious to me.
Re: The 50 million dollar lie
#40Earlier quoted context omitted.
"Whereas to me it looks like it removes visual noise to show an actual trend." Except for that in this case the noise is more important than the trend. Think about it, if you're firing or sanctioning perhaps 30%+ of teachers each year for no reason, then only complete morons would go into teaching. It's the same as airport security, where a .1% false positive rate is unacceptable, whereas a 75% false negative rate is…
Currently the number of public school teachers fired or sanctioned per year is much less than 0.1%. Which is a bit different than 30%. So I'm thinking they're not too worried about this.
[deleted]