Flawed Algorithms Are Grading Millions of Students’ Essays • Hacker News Board

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#81

Post by danenania » Fri, Aug 30, 2019, 1:21 AM UTC

Spivak wrote: post #26

drngdds wrote: post #9

This is my first time learning that AI-graded essays are a thing. Am I the only one who thinks that's insane? I feel like you'd probably have to have an AGI to meaningfully evaluate an essay.

In a forum of CS people I'm surprised this is one of the top opinions. Our field is full of super surprising results like this -- that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it. Like this kind of thing should be cool , not insane. I mean wasn't it cool in your AI class when you learned that DFS could play Mario if…

"...that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it"

That only really shows that the humans they're training on are terrible at grading essays.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#82

Post by dmoy » Fri, Aug 30, 2019, 1:22 AM UTC

harry8 wrote: post #34

VikingCoder wrote: post #15

My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.) One question she had to grade was essentially, "What's something you want your teacher to know about you?" It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.) She sa…

Some of these will be 100% true as well. But don't make the mistake that there are no kids who go for shock value or are wantonly manipulative when they know it can't come back to them. So how many are true and how many false? I have no clue. Literally none. And no it doesn't make me feel any better about the screams of existential agony even if that were a low percentage. Could be high too.

For the not eating, it's pretty easy to get data. It's like 1 in 5 children live in food-insecure households in the US and maybe 1 in 20 of those very insecure, so not eating before school provided lunch is common enough that if you're grading tons of papers you'll run into kids like that.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#83

Post by garmaine » Fri, Aug 30, 2019, 1:24 AM UTC

topologistics wrote: post #47

Hitton wrote: post #27

Earlier quoted context omitted.

>So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower It's quite funny how are some people manipulated to think that society and especially education is somehow biased against minorities or women when opposite is true[1][2]. [1] https://www.studyinternational.com/news/record-high-numbers-... [2] https://www.independent.co.uk/news/world/ame…

The idea that black folks and females score dramatically lower on anything is a commonly held racist opinion that is foundational for the continuation of institutional oppression. When you assume the worst about people it ends up hindering their personal progress.

That's not what the article is saying, at all. There ARE general differences in style and word choice between minority groups or women and the average white male in writing. The corpus of training examples used in making this AI grader are at least biased towards the average white male. When the AI grades an essay harshly, it is not saying "this essay by a black woman is bad writing", it is saying "this essay differes from my training set by an aggregate score of ____", then sorts those results and (maybe) applies a curve.

An essay could be different from the reference standard because the standard is an example of good writing and the essay is not. Or it can be different because the author has a cultural, regional, gender, or developed background that imparts a different style than anything in the training corpus. Mistaking the two is very, very bad.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#84

Post by userbinator » Fri, Aug 30, 2019, 1:26 AM UTC

I've heard stories from others in the industry of companies using tools like this on their human-facing documentation and requiring a certain score from them. Imagine using Microsoft Word's spelling and grammar checker, not being able to add or override its decisions (without following an extremely lengthy and bureaucratic process), and being required to have less than X "defects" per 100 words. Naturally, this results in documentation that is perfectly grammatical and free of spelling errors, but verbose, full of unusual phrasing, and next to useless for its actual purpose of informing a human.

Grading students' code using a machine is not such a bad idea in contrast, because in that case there is [1] no exceptions possible in a programming language, [2] the machine (compiler) has to understand it anyway, and [3] it does save time verifying correctness. But communication in a human language really needs to be assessed by humans. Anyone who thinks "AI" can accurately assess human language is either severely delusional, or trying to make $$$ from it.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#85

Post by [deleted] » Fri, Aug 30, 2019, 1:27 AM UTC

[deleted]

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#86

Post by 40acres » Fri, Aug 30, 2019, 1:27 AM UTC

The sooner we get it out of our heads that this education system of ours is a meritocracy the closer we’ll get to actually creating a quality universal system.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#87

Post by hydrox24 » Fri, Aug 30, 2019, 1:28 AM UTC

Hitton wrote: post #27

jakear wrote: post #3

“In most machine scoring states, any of the randomly selected essays with wide discrepancies between human and machine scores are referred to another human for review”. And “between 5 to 20 percent” of essays are randomly selected for human review. So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower, but are not selected for human review,…

>So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower It's quite funny how are some people manipulated to think that society and especially education is somehow biased against minorities or women when opposite is true[1][2]. [1] https://www.studyinternational.com/news/record-high-numbers-... [2] https://www.independent.co.uk/news/world/ame…

> It's quite funny how are some people manipulated to think that society and especially education is somehow biased against minorities or women when opposite is true

I think you have misunderstood the parent, who is asserting that the machines scoring these essays typically give lower scores to members of these demographics. This does not, by itself, mean the entire system is biased against those groups.

Whether or not the groups are, overall, systematically benefitted or harmed is not relevant to the injustice this article says exists.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#88

Post by [deleted] » Fri, Aug 30, 2019, 1:36 AM UTC

Spivak wrote: post #30

gibolt wrote: post #17

Earlier quoted context omitted.

AI has a lot more potential for bias than humans. It depends on the input data which is likely heavily biased based on other data set results like face detection. It will only amplify any small bias present in the data.

It's amazing to see how the general opinion of CS people has completely shifted in the last few years from "algorithmic scoring is important in removing the bias from human graders" to the exact opposite.

[deleted]

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#89

Post by dredmorbius » Fri, Aug 30, 2019, 1:39 AM UTC

CobrastanJorji wrote: post #29

sjg007 wrote: post #22

Earlier quoted context omitted.

They should be mandatory reporters at least in the USA.

School contractors are mandatory reporters, but I suspect that may not qualify.

Depending on the state, yes, they are, for 11 states.

https://www.childwelfare.gov/topics/systemwide/laws-policies...

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#90

Post by lopmotr » Fri, Aug 30, 2019, 1:39 AM UTC

I wish this machine bias wasn't always presented in such divisive terms as race and "disadvantaged groups". It can affect anybody. If you happened to develop a writing style that looks like typical bad essay writers' style, then you could be hurt by bias in the grading.