What are teachers but flawed Algorithms?
Flawed Algorithms Are Grading Millions of Students’ Essays
161–170 of 302 posts
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#162> the engines also focus heavily on metrics like sentence length, vocabulary, spelling, and subject-verb agreement... The systems are also unable to judge more nuanced aspects of writing, like creativity.
This reminds me of a wonderful essay/speech by Stephen Fry on the harm done by pedantry. I also feel that schools focus so much on a single structure of essay writing and similarly take the joy out of language.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#163> Utah has been using AI as the primary scorer on its standardized tests for several years. “It was a major cost to our state to hand score, in addition to very time consuming,” said Cydnee Carter, the state’s assessment development coordinator. The automated process also allowed the state to give immediate feedback to students and teachers, she said. Yes, education takes time and costs money. Yes, not educating is b…
wait, you get the score in real time? Like some kind of objective function you can train a machine to maximise on?
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#164Earlier quoted context omitted.
The US has federal child abuse mandatory reporting requirement laws which include teachers and school staff and personnel, as well as additional state requirements which vary but include, for 11 states, faculty, staff, and volunteers at public or private higher education institutions. Computer and IT professionals are also covered in cases. Faculty, administrators, athletics staff, or other employees and volunteers a…
so ... nobody wonders about the obvious rammification: then any ML scoring systems ... must detect child abuse signals!
Hey uh, that actually seems valuable.
I'd believe that ML could spot abuse that humans miss pretty well from signals like non-overt references in homework and school records, if one could come up with an adequate training set.
Much more likely than teaching ML to score reasoned and creative activity in any reasonable way.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#165Why are algorithms grading essays in the first place?
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#166Earlier quoted context omitted.
so ... nobody wonders about the obvious rammification: then any ML scoring systems ... must detect child abuse signals!
Hey uh, that actually seems valuable. I'd believe that ML could spot abuse that humans miss pretty well from signals like non-overt references in homework and school records, if one could come up with an adequate training set. Much more likely than teaching ML to score reasoned and creative activity in any reasonable way.
What do you think the false positive rate is likely to be?
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#167> Utah has been using AI as the primary scorer on its standardized tests for several years. “It was a major cost to our state to hand score, in addition to very time consuming,” said Cydnee Carter, the state’s assessment development coordinator. The automated process also allowed the state to give immediate feedback to students and teachers, she said. Yes, education takes time and costs money. Yes, not educating is b…
When I was a child I was obsessed with the "grade level" function in Microsoft Word. It was a preference you could enable on spell check to tell you the "grade level" of your writing. Every essay I wrote, I'd always force myself to reach the max "12.0" grade level. While writing I'd struggle over word choice, sentence structure, rearranging paragraphs, working on my tone etc, all in pursuit of the 12th grade way to p…
You just ruined my childhood
Thanks.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#168Earlier quoted context omitted.
Hey uh, that actually seems valuable. I'd believe that ML could spot abuse that humans miss pretty well from signals like non-overt references in homework and school records, if one could come up with an adequate training set. Much more likely than teaching ML to score reasoned and creative activity in any reasonable way.
What do you think the false positive rate is likely to be?
Depends upon what false negative rate you're willing to tolerate. ;) And I don't know how good of a signal there is. This is pure handwaving.
But this type of thing seems like the exact kind of spooky correlation that ML is good at spotting compared to humans.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#169This is a great example of why it's grossly irresponsible for members of the ML community to talk about how AGI is just around the corner. In addition to the fact that we have no idea whether this is true, it primes a naive public for believing that technologies like this are worth the tradeoff.
"People worry that computers will get too smart and take over the world, but the real problem is that they're too stupid and they've already taken over the world."
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#170> Utah has been using AI as the primary scorer on its standardized tests for several years. “It was a major cost to our state to hand score, in addition to very time consuming,” said Cydnee Carter, the state’s assessment development coordinator. The automated process also allowed the state to give immediate feedback to students and teachers, she said. Yes, education takes time and costs money. Yes, not educating is b…
When I was a child I was obsessed with the "grade level" function in Microsoft Word. It was a preference you could enable on spell check to tell you the "grade level" of your writing. Every essay I wrote, I'd always force myself to reach the max "12.0" grade level. While writing I'd struggle over word choice, sentence structure, rearranging paragraphs, working on my tone etc, all in pursuit of the 12th grade way to p…
The fact that you were literally still a child when this happened, but automated grading is being foisted on us by grown adults who are ostensibly professionals, says a lot about the situation.