Flawed Algorithms Are Grading Millions of Students’ Essays

Post by crankylinuxuser » Fri, Aug 30, 2019, 11:09 AM UTC

wickedsickeune wrote: post #191

clay_the_ripper wrote: post #152

Earlier quoted context omitted.

Wow this is pretty shocking. I can understand using automated systems for something like math problems, it makes sense. There’s (usually) one right answer. But essays? This should be banned.

I have often solved many hard math problems with very unconventional solutions (eg geometrical proof for algebraic problems). Trust me, a piece of software is decades away from being able to accurately determine the future of children and massively impact their self esteem / trust in society.

Post by C1sc0cat » Fri, Aug 30, 2019, 11:33 AM UTC

dagw wrote: post #183

piokoch wrote: post #181

Earlier quoted context omitted.

Frankly, I can't believe what I am reading. The idea that some "AI" grades essays automatically is idiotic and has nothing to do with education. Where is the place for discussions? Where is the place for ideas confrontation? Where is the place for writing style development? How this AI is supposed to grade things like repetitions (that can be either good rhetorical tool or a mistake, depending on context), etc? Who t…

My guess is that soon or later people will learn how to game that AI. Already been done. http://lesperelman.com/writing-assessment-robo-grading/babel... Here's a sample essay that is complete nonsense and got a perfect score on the GRE. http://lesperelman.com/wp-content/uploads/2015/12/6-6_ScoreI...

Post by DoctorOetker » Fri, Aug 30, 2019, 11:38 AM UTC

dvdbloc wrote: post #175

Baeocystin wrote: post #158

Earlier quoted context omitted.

Wait 'til you see a kid in tears because the math answer they submitted was supposed to equal zero, but the algorithms behind the scenes are so bad that the float math failed the equality check. Note: This is not hyperbole, I have seen this exact scenario more than once. There may be a place for a well-designed one, but if it exists, I've never seen it.

I know this is just my experience but I can confirm the automated math scoring system I was using in a large US university in 2012 had bugs where many times I would enter a complex solution with fractions and it would tell me I was wrong and the correct response was some other form of the same equivalent fraction... Talk about frustrating after pouring over a question for 10 minutes.

Post by dagw » Fri, Aug 30, 2019, 11:46 AM UTC

raxxorrax wrote: post #211

dahart wrote: post #133

> Utah has been using AI as the primary scorer on its standardized tests for several years. “It was a major cost to our state to hand score, in addition to very time consuming,” said Cydnee Carter, the state’s assessment development coordinator. The automated process also allowed the state to give immediate feedback to students and teachers, she said. Yes, education takes time and costs money. Yes, not educating is b…

You have automated systems that rate essays without any human actually reading them? Kids, forget everything you know because crime does indeed pay off. Best grades will be reserved for those that try to cheat this system however it is implemented. Botting your essays is the way to go in the 21st century.

Post by UweSchmidt » Fri, Aug 30, 2019, 11:47 AM UTC

clay_the_ripper wrote: post #152

dahart wrote: post #133

> Utah has been using AI as the primary scorer on its standardized tests for several years. “It was a major cost to our state to hand score, in addition to very time consuming,” said Cydnee Carter, the state’s assessment development coordinator. The automated process also allowed the state to give immediate feedback to students and teachers, she said. Yes, education takes time and costs money. Yes, not educating is b…

Wow this is pretty shocking. I can understand using automated systems for something like math problems, it makes sense. There’s (usually) one right answer. But essays? This should be banned.