Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 50 million dollar lie

garyrubinstein.teachforus.org

Post reply on HN
Print view View original

81–90 of 159 posts

Re: The 50 million dollar lie

#81

Post by derpmaster »

This is the corporatist way. Everything is measured so it can be 'tweaked' by managers and technocrats because school is a business now.

Having a bad teacher to me doesn't matter when any kid can turn on his phone or laptop and watch free MIT lectures, free Stanford lectures, free Khanacademy lectures, free Android and iOS developer courses all over youtube, ect.

Gates obsession with teacher metrics would be better spent on building an entirely free university with worldwide standardized credits, so somebody in Pakistan can have their education credentials accepted should they start working overseas instead of being forced to drive a cab around Montreal while spending thousands to retake what he/she already learned.

Re: The 50 million dollar lie

#82

Post by tmoertel »

feral wrote: post #7

I know nothing about the domain of teacher measurement, and have no opinion on it. But the first chart in the blog post, with the big mass of blue on it, is surely not the strong evidence of weak correlation, that the author is making it out to be? There could still be a strong correlation in that data, even if it looks like a blob of blue - because there are so many data points on that chart, that we can no longer t…

Important expectation-calibration note: I'm not making any claims about teacher performance, the usefulness of the value-added idea, or any lofty topics in general. I'm just trying to figure out how to recreate the first graph in the initial posting and get a closer look at the data behind it.

I grabbed the data set [1], loaded it into R, and tried to recreate the first graph in the original post. Observations:

1. The value-added scores are quantized so, as some people have suggested, it's hard to see the true density relationships in the original plot. There's a lot of overlapping data points.

2. I re-plotted the data using jittering to more accurately show the density distribution. I also added a GAM smoother to show the trend. Result: https://docs.google.com/file/d/0B2Tqdso8AsFNS3lWQTdPUGpHZlk/...

3. I also plotted the change in each teacher's value-added score from one year to the next. The resulting histogram: https://docs.google.com/file/d/0B2Tqdso8AsFNeUNiM3hHUGwzdlk/...

4. If we model the relationship as next_year_value_added = this_year_value_added + random_error, here's the fitted model:

    Call:
    lm(formula = va_0910 ~ va_0809, data = ds_cmp)

    Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
    (Intercept) 0.009157   0.002315   3.956 7.67e-05 ***
    va_0809     0.820085   0.017646  46.475
If you want to see exactly what I did, I threw my R code and the data sets up on GitHub: https://github.com/tmoertel/nyva-cursory

[1] http://www.ny1.com/content/top_stories/156599/now-available-...

Re: The 50 million dollar lie

#83

Post by WalterBright »

colanderman wrote: post #59
WalterBright wrote: post #38

There is no objective way to measure teacher performance. Any evaluation method that can be written as a list of rules can and will quickly be gamed. The thing is, it's easy to find out who the best teachers are. Simply ask the students, parents, and staff. They all know who the good ones and the bad ones are - and that can't be gamed.

Your claim makes no sense. If the evaluation method measures what we want teachers to do, then it, by definition, can't be "gamed": if the measurement is high, then the teacher is doing what we want. Are you claiming that there's something special about teaching that makes teacher performance much more difficult, or even impossible, to quantify? If so, what is the point of teaching if one cannot measure results? Your…

I don't think it's so easy to manipulate a classroom full of students into thinking you're a great teacher when you're not. Remember, you're in front of them several hours a day, 5 days a week.

Re: The 50 million dollar lie

#85

Post by jcampbell1 »

kenjackson wrote: post #39
pdonis wrote: post #4

Earlier quoted context omitted.

You're missing the point. The author agreed that there is a slight positive correlation in the data. But the averaging makes it look like there is a strong positive correlation in the data, which there isn't. In other words: without the averaging, you are seeing the full implications of the data: slight positive correlation, but a lot of scatter--i.e., not much meat there. With the averaging, you are seeing only the…

Huh? The averaging doesn't give you only the positive correlation. It simply averages it. If the correlation was negative, you'd see that just as plainly. Averaging doesn't change the trend -- it just removes the volatility from the graph and makes the graph more readable (what's the density at y=x? I suspect it's a lot denser than he'd have you believe, but you can't tell from the chart at all). Sure, you could add…

You are missing the connection between these statistics and school HR policy. Sure there is a small positive correlation, but at what cost should we extract this benefit? Given this data, should we use value-add as a major component of teacher evaluations, and ultimately compensation/terminations?

Giving employee evaluations that are 5% performance and 95% random chance is a recipe for an organization that no capable intelligent person would consider for a career.

Re: The 50 million dollar lie

#86

Post by spikels »

vy8vWJlco wrote: post #36
feral wrote: post #7

I know nothing about the domain of teacher measurement, and have no opinion on it. But the first chart in the blog post, with the big mass of blue on it, is surely not the strong evidence of weak correlation, that the author is making it out to be? There could still be a strong correlation in that data, even if it looks like a blob of blue - because there are so many data points on that chart, that we can no longer t…

A heatmap would be more revealing, but I'd eat my hat if it weren't a standard distribution (and I had a hat). It's nature (but hey, anything's possible). The scales are only slightly skewed, but we're not talking logarithms. The merit of the scales ("value-added") on the other hand, is very questionable. But the relation, whatever it is describing, is pretty weak since the ellipse isn't very irregular (it's only "sl…

Sorry don't mean to belabor this point but I think "standard distribution" (changed from standard deviation) is still incorrect. "Standard" referers to standardized parameters (i.e. mean, variance) as in "standard normal" which has a mean of zero and variance of one (and standard deviation of one). Certainly not the case here. Maybe you mean "bivariate normal with positive correlation"?

http://en.wikipedia.org/wiki/Multivariate_normal_distributio...

Interesting thing is the (positive) means and variance seem similar which is consistent with underlying random variable being the same.

Re: The 50 million dollar lie

#87

Post by acslater00 »

As far as I can tell, the author breaks the universe of teachers into 5% groups and then proceeds to show that the percentile groups are ordinally stable between these two years. In other words, the top scoring group in year 1 beat all other groups in year 2, and so forth, with perhaps some slight shuffling in the absolute middle of the distribution [hard to see on the graph].

This is supposed to be evidence that the correlation is weak??

Re: The 50 million dollar lie

#88

Post by vy8vWJlco »

spikels wrote: post #86
vy8vWJlco wrote: post #36

Earlier quoted context omitted.

A heatmap would be more revealing, but I'd eat my hat if it weren't a standard distribution (and I had a hat). It's nature (but hey, anything's possible). The scales are only slightly skewed, but we're not talking logarithms. The merit of the scales ("value-added") on the other hand, is very questionable. But the relation, whatever it is describing, is pretty weak since the ellipse isn't very irregular (it's only "sl…

Sorry don't mean to belabor this point but I think "standard distribution" (changed from standard deviation) is still incorrect. "Standard" referers to standardized parameters (i.e. mean, variance) as in "standard normal" which has a mean of zero and variance of one (and standard deviation of one). Certainly not the case here. Maybe you mean "bivariate normal with positive correlation"? http://en.wikipedia.org/wiki/M…

Yes, you are right. I wasn't being especially rigorous, and technically correct is the "best kind" of correct. :)

Re: The 50 million dollar lie

#90

Post by bramcohen »

Looking at the correlation in students between year to year makes clear what's going on - http://garyrubinstein.teachforus.org/2012/03/06/analyzing-re...

There's such a high correlation between an individual student's score from one year to the next, and the number of data points is so small, that trying to tweeze out the teacher's contribution in a statistically meaningful way is fairly hopeless.

What I'd like to see are correlates with a teacher's own scores on the tests of the material they're supposed to be teaching (oh you think they all can get perfect scores easily? Hah!) and whether teachers banned from giving homework do any worse, and whether dumping the enforced curriculum in favor of letting students study at their own pace makes any difference. Given how little what the teachers actually do seems to make, it would be logical to at least dump the things which make school dull and unpleasant.

Post reply on HN