Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U-M finds students with alphabetically lower-ranked names receive lower grades

record.umich.edu

Post reply on HN
Print view View original

111–120 of 284 posts

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#111

Post by JadeNB »

dotnet00 wrote: post #9

Yep, I noticed this with myself too when I first did some grading a few months ago. There was also the factor that the ones I graded initially did not make certain mistakes or answered in expected ways, such that when I did encounter unexpected answers/mistakes, I had to go back and rethink the grading on the papers I had graded previously. Eg if someone answered in a way that made me think an answer I considered inc…

> I only had to deal with a small class, so backtracking was doable and I graded the papers in whatever shuffled up order they were turned in, otherwise there would have definitely been a bias.

Grading papers in submission order just introduces a different bias, though.

(For what it's worth, I'm in the same boat and I do the same, because I don't trust my ability to give the papers any true random sorting by hand, so I take the very weak randomization that the submission order gives me.)

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#112

Post by StefanBatory »

I have an surname that's alphabetically low. Even at uni amount time I went to class and came out empty-handed as my teacher didn't score my assignement on time (at my uni 90% we have oral discussion about it) and I have to come next week while others don't are way too high.

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#113

Post by trescenzi »

noodlesUK wrote: post #6

At my university, almost all of our marking was pseudonymised. We were assigned a random candidate number at the beginning of each year, and that is what went on our important papers/exams. The less important coursework often didn’t bother with this, and used our student numbers instead, but the general idea was the same. We didn’t put our names on any of our work other than our dissertation (and a few trivial assign…

Wouldn't a possible outcome here though be that it just randomly reduces grades instead of reducing them in a way that's related to the students? If the issue is the sorting the random candidate numbers would still be sorted. It solves the problem of bias related to the individual but it doesn't solve the problem of bias related to the way that the submissions are sorted.

A random identifier coupled with a random sort order seem like the way to go here.

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#114

Post by pjdesno »

lqet wrote: post #41

I work in academia. When we grade exams, the order of the exams on the stack is the order in which they were collected in the room (people can sit wherever they like). For grading, we are usually 5 people in a single room, and everyone grades a specific exercise for consistency. The exams are getting shuffled heavily , with everyone just grabbing stacks, looking for exams where "their" exercise was not yet graded, an…

There are all sorts of good ways to avoid these biases. I use the same practice described above for paper exams, and grading order for eg question 2 may be affected by score on question 1, but it won’t be affected by name or ID number.

If you use Canvas or Gradescope with the default settings, it’s almost impossible to avoid this sort of bias.

Worse yet, in Gradescooe you’re strongly steered towards grading with a fixed “rubric” with specific points off for each of N pre-defined errors, allowing grading to be done by TAs with little more knowledge than the students themselves, resulting in scores which have little relationship to the quality of the student answer.

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#115

Post by throwaway35777 »

13of40 wrote: post #37
bluGill wrote: post #10

Earlier quoted context omitted.

> it would mean that the graders are just getting tired/lazy/inattentive the further they get in their stack of papers Or maybe they are getting better / more picky. I know in code reviews I often pass a few and then notice something that I realize was also wrong in previous reviews I allowed, but later reviews that day (week?) will not allow that.

I've participated in day-long and multi-day interview events for job candidates, and I see the same effect. At the beginning you don't have a frame of reference and you're more likely to question your own decision or give someone the benefit of the doubt, but by the end you're far more systematic, plus a little bit numb to the effect your decision is having.

> by the end you're far more systematic, plus a little bit numb to the effect your decision is having

Maybe decision fatigue is supposed to bias humans toward the optimal solution for the fiancee problem [1].

[1] https://en.m.wikipedia.org/wiki/Secretary_problem

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#116

Post by sdwr »

StevenXC wrote: post #25
zdw wrote: post #15

As someone whose initials are Z and W, I tend to notice alpha sort a lot. Asking a friend whose initials are A and B about this, it's not something they ever noticed. I haven't noticed a grading/ranking difference, but far more frequently I'll hear that "oh, we ran out of item/time/etc. before we got to you", which has made me much more sensitive to issues of planning/organization.

Like most inequities, those who are in the benefiting group frequently don't realize that privilege.

They realize (bring form to, make real) them, but don't realize (understand) them

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#117

Post by jcla1 »

cvwright wrote: post #101
lqet wrote: post #41

I work in academia. When we grade exams, the order of the exams on the stack is the order in which they were collected in the room (people can sit wherever they like). For grading, we are usually 5 people in a single room, and everyone grades a specific exercise for consistency. The exams are getting shuffled heavily , with everyone just grabbing stacks, looking for exams where "their" exercise was not yet graded, an…

My strategy was to, like you said, grade problem by problem. Then for each problem, first find all those who got full marks. Then group the others into piles based on what mistakes they made. This ensures that everyone who made the same mistake(s) gets the same grade. It also tends to shuffle the order of the exams after every problem. Obviously you don’t need this strategy for simple multiple choice questions, and i…

This sounds like an organisational nightmare to be honest. You'd be going through the pile of exams multiple times (at least twice) and what do you do if there are multiple mistakes that are common in a single exam question?

Also: if you're sorting into "mistakes piles" for single exercises, how can you parallelise marking of separate and independent questions?

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#118

Post by Zancarius »

smogcutter wrote: post #107
V__ wrote: post #88

Earlier quoted context omitted.

A teacher friend of mine always goes through his stack twice. Once to correct all mistakes and a second time to write down points. As you said, once you have seen all mistakes you know how "bad" of a mistake it actually is.

> As you said, once you have seen all mistakes you know how "bad" of a mistake it actually is. Crucially, this is not quite what the poster said. It’s not about stack ranking students against each other. Say every paper makes the same subtle mistake, and you only notice it halfway through the pile. Unless you go back through them all, you’ll unfairly grade the later entries more harshly.

> It’s not about stack ranking students against each other.

It's not, but it sort of has that effect, albeit indirectly, and definitely unfairly.

Re: U-M finds students with alphabetically lower-ranked names receive lower grades

#120

Post by ghaff »

__MatrixMan__ wrote: post #104
noodlesUK wrote: post #6

At my university, almost all of our marking was pseudonymised. We were assigned a random candidate number at the beginning of each year, and that is what went on our important papers/exams. The less important coursework often didn’t bother with this, and used our student numbers instead, but the general idea was the same. We didn’t put our names on any of our work other than our dissertation (and a few trivial assign…

I think I get better feedback when the teacher knows who I am. Grades are secondary.

I'm not sure exam grades at the university level are really the place to get useful feedback beyond grades.
Post reply on HN