Flawed Algorithms Are Grading Millions of Students’ Essays • Hacker News Board

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#101

Post by mlthoughts2018 » Fri, Aug 30, 2019, 1:57 AM UTC

_delirium wrote: post #24

mlthoughts2018 wrote: post #21

Earlier quoted context omitted.

Hmmm. I also work in AI, in fact professionally in information retrieval and NLP. I disagree strongly with what you say. Basic topic summarization and keyword / named entity extraction on unstructured sources of text works reasonably well. It’s easy to modify BERT and GPT on smaller problems, language classification is borderline totally solved by extremely easy to train neural network models. I still agree that auto…

It's completely possible I'm not fully up on recent progress, especially since a bunch of stuff seems to have moved in the past 6 months. But I haven't seen any general models that can solve open-domain problems, without specifically retraining on each domain. Do you have any pointers? E.g. a single pretrained BERT model that can reliably extract topics from: tweets, paragraphs from 19th-century novels, mathematics j…

In production problems that I work on, we don’t even really use things from within the past year. These problems are just incredibly well-solved with fairly vanilla LSTM networks from 2-3 years ago. Enough so that while it’s probably premature for fully automated essay grading, it’s not _crazy_ to make a product from models trained to solve this problem.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#102

Post by crooked-v » Fri, Aug 30, 2019, 1:58 AM UTC

aidenn0 wrote: post #43

crispcarb wrote: post #10

Because a (likely unsophisticated) algorithm is grading the essays, there's probably a deterministic method to do score well. This seems like a terrible idea. It's not a stretch to imagine the opportunity for nefarious behavior this allows - think of the recent college admission scandals, and how happy they'd be to have a guise of algorithmic indifference'. If used long-term, it could offer a big advantage to the wea…

Surely there's a deterministic method to score well on the math questions?

An essay is to a math problem as a proof is to a grammar problem.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#103

Post by lonelappde » Fri, Aug 30, 2019, 1:59 AM UTC

Hitton wrote: post #27

jakear wrote: post #3

“In most machine scoring states, any of the randomly selected essays with wide discrepancies between human and machine scores are referred to another human for review”. And “between 5 to 20 percent” of essays are randomly selected for human review. So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower, but are not selected for human review,…

>So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower It's quite funny how are some people manipulated to think that society and especially education is somehow biased against minorities or women when opposite is true[1][2]. [1] https://www.studyinternational.com/news/record-high-numbers-... [2] https://www.independent.co.uk/news/world/ame…

That link talks about how black women are attending college in highest proportion due to extremely recent growth. It also says that despite their recently high educational attainment, black women are still underrepresented by a factor of 2 in private sector jobs, as compared to their college graduation rates.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#104

Post by chucksmash » Fri, Aug 30, 2019, 2:00 AM UTC

drngdds wrote: post #9

This is my first time learning that AI-graded essays are a thing. Am I the only one who thinks that's insane? I feel like you'd probably have to have an AGI to meaningfully evaluate an essay.

I'm surprised people are surprised by it. I guess it just hasn't gotten talked about it a lot? When I took the GRE in 2011 the rule was that my essay would be graded by one human and one automated grader, and a second human would become involved if the computer and the human differed by one point or more iirc.

Maybe nobody really makes a big deal about it because it is pretty much irrelevant anywah. Applicants provide a letter of intent that the grad dept people can, y'know, actually read for themselves, so I think unless you totally bombed the writing section nobody cared.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#105

Post by crooked-v » Fri, Aug 30, 2019, 2:01 AM UTC

lopmotr wrote: post #90

I wish this machine bias wasn't always presented in such divisive terms as race and "disadvantaged groups". It can affect anybody. If you happened to develop a writing style that looks like typical bad essay writers' style, then you could be hurt by bias in the grading.

Anyone can be hurt by bias, but minority groups are the obvious ones to be most likely statistically affected by it, making them the most obvious red flag for this kind of situation.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#106

Post by strken » Fri, Aug 30, 2019, 2:04 AM UTC

Spivak wrote: post #26

drngdds wrote: post #9

This is my first time learning that AI-graded essays are a thing. Am I the only one who thinks that's insane? I feel like you'd probably have to have an AGI to meaningfully evaluate an essay.

In a forum of CS people I'm surprised this is one of the top opinions. Our field is full of super surprising results like this -- that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it. Like this kind of thing should be cool , not insane. I mean wasn't it cool in your AI class when you learned that DFS could play Mario if…

This is sort of like discovering the Excel spreadsheet at the heart of a system responsible for handling hundreds of millions of dollars of transactions for your bank.

Yeah, it's cool, but what about your savings account?

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#107

Post by LeonB » Fri, Aug 30, 2019, 2:08 AM UTC

Hitton wrote: post #27

jakear wrote: post #3

“In most machine scoring states, any of the randomly selected essays with wide discrepancies between human and machine scores are referred to another human for review”. And “between 5 to 20 percent” of essays are randomly selected for human review. So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower, but are not selected for human review,…

>So the takeaway is that if you’re one of the 80-95% of (typically black or female) people who the machine scored dramatically lower It's quite funny how are some people manipulated to think that society and especially education is somehow biased against minorities or women when opposite is true[1][2]. [1] https://www.studyinternational.com/news/record-high-numbers-... [2] https://www.independent.co.uk/news/world/ame…

The links you provided don’t show what you claim.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#108

Post by aaron695 » Fri, Aug 30, 2019, 2:22 AM UTC

Half of Vice is garbage and it's growing.

They seem to have figured out being political gets them clicks over substance. So much for the days they just reported stories other media wouldn't cover.

Any sources backing anything here?

Is NLP actually used to primarily grade papers anywhere in the USA?

It seems way beyond current technology.

Checking if a human marker doesn't need their score checked, perhaps could possibly be implemented.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#109

Post by fzeroracer » Fri, Aug 30, 2019, 2:27 AM UTC

lopmotr wrote: post #90

I wish this machine bias wasn't always presented in such divisive terms as race and "disadvantaged groups". It can affect anybody. If you happened to develop a writing style that looks like typical bad essay writers' style, then you could be hurt by bias in the grading.

If an image processing algorithm fails to recognize black people or worse, profiles them, how else should this be described but in terms of race?

If you don't talk about the actual problem, how can you possibly expect to solve it?

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#110

Post by bendbro » Fri, Aug 30, 2019, 2:31 AM UTC

shaggyfrog wrote: post #48

empath75 wrote: post #28

Earlier quoted context omitted.

‘Poor sentence structure and grammar, 1 point out of five. Sorry your daddy touches you.’

Punch up, not down.

Do you suspect this is a one off mistake or do you think you make repeated mistakes of this kind? I'm always worried I'm doing some particular thing, interpersonally, repeatedly, but I fail to notice.