Would it be possible to simply accept that this exists and automatically unskew the grades after marking?
U-M finds students with alphabetically lower-ranked names receive lower grades
241–250 of 284 posts
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#242Earlier quoted context omitted.
Everything in this thread just randomizes who doesn't get graded fairly.
Is there a better solution? It's not for teachers to be perfect. Since that's not possible, it's not a solution.
Automatically unskew the results after grading based on this finding?
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#243Earlier quoted context omitted.
Is there a better solution? It's not for teachers to be perfect. Since that's not possible, it's not a solution.
No, the solution is for the scoring to be handled by software that doesn't exist yet. Some things have easy, objective measures of correctness. STEM is mostly this way. Others, your humanities et al, are fairly subjective. You could probably cover most of this with an LLM, and access to a large body of graded material for a given course, provided said material was graded fairly. Generating that data would be time con…
LLMs are not going to be a solution. LLMs have absolutely no concept of truth.
And not everything has an objective solution. Even those that do often have a process associated with them and factoring in that work/process is an important part of grading. Reducing that subjective grading process to only objective solutions being right is grossly reductive and disproportionately punishes students who have the process right and understand the material but make small errors. That's exactly what you don't want to do.
---
Instead the solution is to make sure each assignment gets multiple eyes on it and in a random order. Then to document biases and trends in biases so that the TAs and professors can be aware of them and mitigate them.
It's a process problem that can only be solved by a process solution. Replacing the graders with technology or reducing problems to a binary right/wrong will never ever solve this and in many cases will end up being more harmful than the biases they claim to solve would be.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#244There's a section of one of the Diary of a Wimpy Kid books that talks about this exact thing. I was reminded of it as soon as I saw the headline. The justification is comes up with is that kids with names at the front of the alphabet sit in the front of the classroom, so they get called on and learn more. It definitely turned some gears in my brain when I first read it as a teen. Here's the relevant page: https://imgur.com/a/6wIx6qg
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#245Earlier quoted context omitted.
My strategy was to, like you said, grade problem by problem. Then for each problem, first find all those who got full marks. Then group the others into piles based on what mistakes they made. This ensures that everyone who made the same mistake(s) gets the same grade. It also tends to shuffle the order of the exams after every problem. Obviously you don’t need this strategy for simple multiple choice questions, and i…
When I was a TA at CMU, we used Gradescope https://www.gradescope.com/ for this. Every exam would be scanned and divided into problems (based on a predefined template - fixed page space for answers). Then, each problem was assigned to a TA. Either there's a predefined rubric, or you create it as you go (-1 point for mistake X, half credit for mistake Y, etc.). There's a pretty slick interface where you just read the…
There’s also open-source software that does the same job at TU Delft: https://zesje.tudelft.nl/
(disclaimer: I briefly worked on the software for my bachelor’s thesis)
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#246Earlier quoted context omitted.
They realize (bring form to, make real) them, but don't realize (understand) them
I hate how much I love this worthlessly picky comment
Love me or hate me, you gotta love me!
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#247Earlier quoted context omitted.
No, the solution is for the scoring to be handled by software that doesn't exist yet. Some things have easy, objective measures of correctness. STEM is mostly this way. Others, your humanities et al, are fairly subjective. You could probably cover most of this with an LLM, and access to a large body of graded material for a given course, provided said material was graded fairly. Generating that data would be time con…
LLMs are not going to be a solution. LLMs have absolutely no concept of truth. And not everything has an objective solution. Even those that do often have a process associated with them and factoring in that work/process is an important part of grading. Reducing that subjective grading process to only objective solutions being right is grossly reductive and disproportionately punishes students who have the process ri…
The LLM can compile verbose prose down to a short summary. If the summaries of each chunk are consistent, then it’s at least structurally well written. Then you grade the summary itself.
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#248Earlier quoted context omitted.
I know a guy who copy/pasted a wikipedia article, in line citations and all, and submitted it for a sociology class and got an A, no notes, nothing.
He “only cheated himself.” :-D
The point is to develop skills and knowledge, so I would agree. Do you disagree?
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#249As someone whose initials are Z and W, I tend to notice alpha sort a lot. Asking a friend whose initials are A and B about this, it's not something they ever noticed. I haven't noticed a grading/ranking difference, but far more frequently I'll hear that "oh, we ran out of item/time/etc. before we got to you", which has made me much more sensitive to issues of planning/organization.
Similar initials, frequently last in line, and same. I wonder if this was the kiln of my patience and acceptance, or if people who road rage and get frustrated with waits are more likely to have earlier lettered names?
I really cannot stand sitting in a car in traffic and my last name starts with an A, interesting!
Re: U-M finds students with alphabetically lower-ranked names receive lower grades
#250I work in academia. When we grade exams, the order of the exams on the stack is the order in which they were collected in the room (people can sit wherever they like). For grading, we are usually 5 people in a single room, and everyone grades a specific exercise for consistency. The exams are getting shuffled heavily , with everyone just grabbing stacks, looking for exams where "their" exercise was not yet graded, an…
> you have to shuffle the order in which you grade these submissions each week, for fairness I don't think this is fair. It's just a more randomly distributed unfairness, rather than by a deterministic factor (like the student's name) 'Fair' would be each student is assessed independently for the work they did, rather than their mark being impacted by how early or late they were marked.
I think an important difference is that when you shuffle them, the unfairness stops being correlated across multiple assignments, so the "aggregate" unfairness over the course of the semester is much lower.