Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flawed Algorithms Are Grading Millions of Students’ Essays

vice.com

Post reply on HN
Print view View original

91–100 of 302 posts

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#91

Post by bumby »

cmroanirgo wrote: post #63
Spivak wrote: post #26

Earlier quoted context omitted.

In a forum of CS people I'm surprised this is one of the top opinions. Our field is full of super surprising results like this -- that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it. Like this kind of thing should be cool , not insane. I mean wasn't it cool in your AI class when you learned that DFS could play Mario if…

I came first in English for my school, many moons ago. Leading up to the finals, I regularly finished ahead of the hard core the English essay people, generally to my amusement. My exam essay responses were generally half the length (sometimes even shorter) than the prodigious writers. Although I've an ok vocabulary, I always made sure I made the right choice of word to hit a specific meaning, rather than choosing wo…

Your post resonated with my first thought on reading the article. I wonder if it would penalize writers with simple declarative sentences.

You know, those average writers like Hemingway /s

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#92

Post by nyxtom »

Good lord what a terrible design. Rather than determine if the writer has a coherent understanding and a complex prompt, the system grades based on writing patterns. This is actually my biggest fear of AI. Deploying wide scale systems like this that have very clear flaws

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#93

Post by dredmorbius »

VikingCoder wrote: post #15

My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.) One question she had to grade was essentially, "What's something you want your teacher to know about you?" It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.) She sa…

The US has federal child abuse mandatory reporting requirement laws which include teachers and school staff and personnel, as well as additional state requirements which vary but include, for 11 states, faculty, staff, and volunteers at public or private higher education institutions. Computer and IT professionals are also covered in cases.

Faculty, administrators, athletics staff, or other employees and volunteers at institutions of higher learning, including public and private colleges and universities and vocational and technical schools (11 States).

https://www.childwelfare.gov/topics/systemwide/laws-policies...

https://www.childwelfare.gov/pubPDFs/manda.pdf

This includes penalties for failure to report in multiple states:

https://www.childwelfare.gov/topics/systemwide/laws-policies...

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#95

Post by [deleted] »

shkkmo wrote: post #64
topologistics wrote: post #47

Earlier quoted context omitted.

The idea that black folks and females score dramatically lower on anything is a commonly held racist opinion that is foundational for the continuation of institutional oppression. When you assume the worst about people it ends up hindering their personal progress.

They do when graded by a computer program as described in the article. Are you disputing the article's veracity? What is your point?

[deleted]

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#96

Post by lonelappde »

jacobwilliamroy wrote: post #72
fucking_tragedy wrote: post #58

Earlier quoted context omitted.

> But don't make the mistake that there are no kids who go for shock value or are wantonly manipulative when they know it can't come back to them. In the US, school funding is based upon standardized test results, and bad results can shut a poorly performing school down. It's drilled into every kid's head that these tests are very important, super strict and if they accidentally mess up, it can ruin their academics,…

As a kid I would go out of my way to fail those tests. The whole curriculum was designed around them, meaning that even if we did score high, any funding gains would just be put towards training us to take the test. I thought the state was holding the school hostage, threatening to cut funds or shut them down if they ever stopped. We never learned anything about civics or American history. Until I was out of highscho…

Your school didn't teach slavery? What state?

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#97

Post by lonelappde »

peteradio wrote: post #49
Spivak wrote: post #26

Earlier quoted context omitted.

In a forum of CS people I'm surprised this is one of the top opinions. Our field is full of super surprising results like this -- that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it. Like this kind of thing should be cool , not insane. I mean wasn't it cool in your AI class when you learned that DFS could play Mario if…

Based on the title alone, how would you feel if you were given bad marks due to flawed black box?

The same as when I get bad marks due to a flawed human and rubric.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#98

Post by lonelappde »

ta8347 wrote: post #50
hnburnsy wrote: post #36

Earlier quoted context omitted.

They did drop the writing but replaced it with an optional essay. Most upper tier schools require applicants to take the essay.

Harvard, MIT, Stanford, Caltech, Princeton, Yale, Dartmouth and many other schools do NOT require applicants to take the SAT essay.

Because they prefer an essay you can buy from a college applications consultant.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#99

Post by ineedasername »

garmaine wrote: post #74
CobrastanJorji wrote: post #29

Earlier quoted context omitted.

School contractors are mandatory reporters, but I suspect that may not qualify.

These would not be school contractors.

Why not? The school contracts with the testing agency that does the grading. Seems like a contractor relationship?

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#100

Post by thekyle »

mherdeg wrote: post #56
Spivak wrote: post #26

Earlier quoted context omitted.

In a forum of CS people I'm surprised this is one of the top opinions. Our field is full of super surprising results like this -- that you don't have to actually understand the text at beyond basic grammar structures to reasonably accurately predict the score a human would give it. Like this kind of thing should be cool , not insane. I mean wasn't it cool in your AI class when you learned that DFS could play Mario if…

When I hear a result like "software which understands basic grammar structures can predict what grade a human would give an essay" I think my views are roughly: * 5% - cool, we could make a company that grades essays * 15% - cool, we could make a company that grades essays and sell our source code to the test-prep industry * 80% - fascinating, it sounds like the exam designers need to reevaluate what they are trying…

Whatever we decide to measure, it needs to scale to millions of essay responses each year in a way such that scores are consistent across entire states or countries. With that in mind I'd imagine it's difficult to do much more than grade on grammar and basic semantics.
Post reply on HN