U-M finds students with alphabetically lower-ranked names receive lower grades
1–10 of 284 posts
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#2> Wang noted that for a small group of graders (about 5%) that grade from Z to A, the grade gap flips as expected
This is critical. Otherwise we could not discount some group (e.g. some ethnicity) disproportionately occupying one end of the alphabet or another.
Super interesting and important finding. I hope this gets wide visibility and universities take a break from politicking to fix the problem - presumably through enforced randomizing.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#3This seems related: https://news.ycombinator.com/item?id=39672111
As in, order matters
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#4> Wang noted that for a small group of graders (about 5%) that grade from Z to A, the grade gap flips as expected This is critical. Otherwise we could not discount some group (e.g. some ethnicity) disproportionately occupying one end of the alphabet or another. Super interesting and important finding. I hope this gets wide visibility and universities take a break from politicking to fix the problem - presumably throu…
Enforced randomization isn't going to fix the problem, it just evenly distributes the problem.
Based on these results, it would mean that the graders are just getting tired/lazy/inattentive the further they get in their stack of papers to grade. That's the problem the needs to be fixed, not the order they get graded in. Enforced randomization is simply a short term alleviation so no student(s) get disproportionately affected by this phenomenon.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#5>One simple fix would be to make random order the default setting.
Fixed in the sense that the bias will be random. Presumably students graded last will still receive lower grades.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#6At my university, almost all of our marking was pseudonymised. We were assigned a random candidate number at the beginning of each year, and that is what went on our important papers/exams. The less important coursework often didn’t bother with this, and used our student numbers instead, but the general idea was the same.
We didn’t put our names on any of our work other than our dissertation (and a few trivial assignments that didn’t impact overall marks). It wasn’t that hard to de-anonymise, but it meant that the system had a bit more integrity.
It’s a really straightforward system to implement and I don’t know why it isn’t done more frequently.
I also think that our VLE sorted assignments by time of submission rather than any identifier.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#7> Wang noted that for a small group of graders (about 5%) that grade from Z to A, the grade gap flips as expected This is critical. Otherwise we could not discount some group (e.g. some ethnicity) disproportionately occupying one end of the alphabet or another. Super interesting and important finding. I hope this gets wide visibility and universities take a break from politicking to fix the problem - presumably throu…
Enforced randomization isn't going to fix the problem, it just evenly distributes the problem. Based on these results, it would mean that the graders are just getting tired/lazy/inattentive the further they get in their stack of papers to grade. That's the problem the needs to be fixed, not the order they get graded in. Enforced randomization is simply a short term alleviation so no student(s) get disproportionately…
In the real world, universities are never going to fix the problem of overworked and underpaid grad students getting tired.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#8>One simple fix would be to make random order the default setting. Fixed in the sense that the bias will be random. Presumably students graded last will still receive lower grades.
It would be less than ideal, but still an improvement over the current situation as long as the order is re-randomized for every assignment, because at least then you'd only be occasionally disadvantaged rather than consistently disadvantaged.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#9Yep, I noticed this with myself too when I first did some grading a few months ago.
There was also the factor that the ones I graded initially did not make certain mistakes or answered in expected ways, such that when I did encounter unexpected answers/mistakes, I had to go back and rethink the grading on the papers I had graded previously. Eg if someone answered in a way that made me think an answer I considered incorrect was actually less wrong.
I only had to deal with a small class, so backtracking was doable and I graded the papers in whatever shuffled up order they were turned in, otherwise there would have definitely been a bias.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#10> Wang noted that for a small group of graders (about 5%) that grade from Z to A, the grade gap flips as expected This is critical. Otherwise we could not discount some group (e.g. some ethnicity) disproportionately occupying one end of the alphabet or another. Super interesting and important finding. I hope this gets wide visibility and universities take a break from politicking to fix the problem - presumably throu…
Enforced randomization isn't going to fix the problem, it just evenly distributes the problem. Based on these results, it would mean that the graders are just getting tired/lazy/inattentive the further they get in their stack of papers to grade. That's the problem the needs to be fixed, not the order they get graded in. Enforced randomization is simply a short term alleviation so no student(s) get disproportionately…
> it would mean that the graders are just getting tired/lazy/inattentive the further they get in their stack of papers
Or maybe they are getting better / more picky.
I know in code reviews I often pass a few and then notice something that I realize was also wrong in previous reviews I allowed, but later reviews that day (week?) will not allow that.