Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The 50 million dollar lie

garyrubinstein.teachforus.org

Post reply on HN
Print view View original

131–140 of 159 posts

Re: The 50 million dollar lie

#131

Post by WalterBright »

yummyfajitas wrote: post #115
DataWorker wrote: post #111

Earlier quoted context omitted.

A few come to mind: cheating (see Atlanta Journal-Constitution series on this) encouraging absenteeism amongst low performers damaging the test materials of low performing students to get their scores excluded getting low performers into IEPs so they don't test, take the alternate or receive accommodations sabotage other teachers in various ways vie for the most favorable students by pulling strings with the administ…

Most of these issues can be addressed simply by completely separating education and evaluation. I.e., teachers take the day off when tests are administered, some bureaucrat shows up and does it instead. Treating damaged test materials/absent students as a score of 0 is the simplest way to prevent hacking the student body. vie for the most favorable students by pulling strings with the administration The most favorabl…

The teachers know who the good students are, just like the students know who the good teachers are.

Re: The 50 million dollar lie

#132

Post by WalterBright »

yummyfajitas wrote: post #115
DataWorker wrote: post #111

Earlier quoted context omitted.

A few come to mind: cheating (see Atlanta Journal-Constitution series on this) encouraging absenteeism amongst low performers damaging the test materials of low performing students to get their scores excluded getting low performers into IEPs so they don't test, take the alternate or receive accommodations sabotage other teachers in various ways vie for the most favorable students by pulling strings with the administ…

Most of these issues can be addressed simply by completely separating education and evaluation. I.e., teachers take the day off when tests are administered, some bureaucrat shows up and does it instead. Treating damaged test materials/absent students as a score of 0 is the simplest way to prevent hacking the student body. vie for the most favorable students by pulling strings with the administration The most favorabl…

The teachers know who the good students are, just like the students know who the good teachers are.

Re: The 50 million dollar lie

#133

Post by WalterBright »

yummyfajitas wrote: post #99
WalterBright wrote: post #80

Earlier quoted context omitted.

You can't specify everything in the rules. For example, I know a case where management decided to hand out bonuses to the staff for minimizing inventory, because inventory costs money. The staff minimized inventory, all right, and got their bonuses. Meanwhile, the production line would regularly get halted because they'd run out of things like 5 cent resistors. It was a disaster. The same company decided to rate prog…

For teachers, the metric is (roughly speaking) "% of students capable of multiplying/dividing numbers up to 4 digits in a standardized test setting". How do you game this metric? The fact that one company used a couple of bad objective metrics doesn't mean all objective metrics are bad. They are used with a great deal of success in many fields. Sales people are paid on commission, traders are paid proportionally to (…

Consider also how athletes game the drug tests - such as Lance Armstrong.

Re: The 50 million dollar lie

#134

Post by [deleted] »

kevinalexbrown wrote: post #54
feral wrote: post #7

I know nothing about the domain of teacher measurement, and have no opinion on it. But the first chart in the blog post, with the big mass of blue on it, is surely not the strong evidence of weak correlation, that the author is making it out to be? There could still be a strong correlation in that data, even if it looks like a blob of blue - because there are so many data points on that chart, that we can no longer t…

Edit: since ColinWright explicitly asked us to indicate experience with hard statistics: Check the link in my profile. I work with very messy data. I've also found that the more experience people have with something, the less confident they are about their expertise so I won't claim to be NotWrong, but the statistics here really aren't hard. It's linear regression with data that appears, cursorily, to be gaussian, cl…

[deleted]

Re: The 50 million dollar lie

#135

Post by DataWorker »

DataWorker wrote: post #130
tmoertel wrote: post #82

Earlier quoted context omitted.

Important expectation-calibration note: I'm not making any claims about teacher performance, the usefulness of the value-added idea, or any lofty topics in general. I'm just trying to figure out how to recreate the first graph in the initial posting and get a closer look at the data behind it. I grabbed the data set [1], loaded it into R, and tried to recreate the first graph in the original post. Observations: 1. Th…

One question would be whether or not to match the files on subject/grade/teacher, or just the teacher. You use subject/grade/teacher level, but you get a different answer if you use aggregated teacher level.

I tried it by teacher and got a correlation of .45.

(I used unweighted average for each teacher/school/year.)

Re: The 50 million dollar lie

#136

Post by tmoertel »

carbocation wrote: post #65

I'm working on reproducing the graph with the data available from the NYT, but I'm not sure on one or two details of the author's method. If someone sees it, please let me know! Things I can't tell so far: 1) How is a "teacher" defined? Just(hash firstname lastname)? If so... 2) When a teacher teaches multiple subjects per year, which one is chosen? Or are they both represented, so that "teacher" actually means (hash…

If you want to see how I recreated the OP's graph, just fork my GitHub repo. It has R code and the data sets, too:

https://github.com/tmoertel/nyva-cursory

Anyway, here are the variables I used to identify comparable observations:

    ## these variables identify comparable teacher observations
    id_vars
So I used subject, grade, school (identified by dbn) and the teacher's first and last names.

Re: The 50 million dollar lie

#137

Post by yummyfajitas »

WalterBright wrote: post #131
yummyfajitas wrote: post #115

Earlier quoted context omitted.

Most of these issues can be addressed simply by completely separating education and evaluation. I.e., teachers take the day off when tests are administered, some bureaucrat shows up and does it instead. Treating damaged test materials/absent students as a score of 0 is the simplest way to prevent hacking the student body. vie for the most favorable students by pulling strings with the administration The most favorabl…

The teachers know who the good students are, just like the students know who the good teachers are.

If the statistical predictor in vam also knows who the good students are, this won't help. All it will do is raise the bar for that teacher.

Re: The 50 million dollar lie

#138

Post by yummyfajitas »

yummyfajitas wrote: post #117
patio11 wrote: post #108

Earlier quoted context omitted.

That is common for pre-moderated comments on blogs rather than evidence of hell banning, fwiw.

My mistake. Apologies to GR if that is the case.

Looks like I overreacted. My post is showing up fine. I often warn against assuming the worst of those you disagree with, and I should have heeded my own warnings.

Re: The 50 million dollar lie

#139

Post by carbocation »

tmoertel wrote: post #136
carbocation wrote: post #65

I'm working on reproducing the graph with the data available from the NYT, but I'm not sure on one or two details of the author's method. If someone sees it, please let me know! Things I can't tell so far: 1) How is a "teacher" defined? Just(hash firstname lastname)? If so... 2) When a teacher teaches multiple subjects per year, which one is chosen? Or are they both represented, so that "teacher" actually means (hash…

If you want to see how I recreated the OP's graph, just fork my GitHub repo. It has R code and the data sets, too: https://github.com/tmoertel/nyva-cursory Anyway, here are the variables I used to identify comparable observations: ## these variables identify comparable teacher observations id_vars So I used subject, grade, school (identified by dbn) and the teacher's first and last names.

Nice, thanks for sharing. I used the same variables with the exception of dbn; my code is here https://gist.github.com/4652968 (I later started applying filters such as 4th-grade teachers only, etc., which is reflected in the gist.)

I was pretty curious how the author specifically munged his data, since then we could put to rest the speculation about the degree of correlation.

Re: The 50 million dollar lie

#140

Post by spikels »

keithpeter wrote: post #46

" In New York City, the value-added score is actually not based on comparing the scores of a group of students from one year to the next, but on comparing the ‘predicted’ scores of a group of students to what those students actually get. The formula to generate this prediction is quite complicated, but the main piece of data it uses is the actual scores that the group of students got in the previous year. " http://ga…

You are correct that if students were assigned in some biased manner it could invalidate any relationship between teacher quality and subsequent student performance.

That is exactly what makes this study interesting. All the students were assigned randomly! Much like in a randomized medical study this allows the establishment of a causal relationship.

Post reply on HN