Live data from Hacker News

Flawed Algorithms Are Grading Millions of Students’ Essays

vice.com

301–302 of 302 posts

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#301

Earlier quoted context omitted.

Why not? The school contracts with the testing agency that does the grading. Seems like a contractor relationship?

School contractor is someone who is hired by the school district on via a contractual relationship. Think temporary teachers, or custodian staff. It’s not a transitive relationship to every employee of every company who has some sort of contract, however small, with a school.

So you're arguing that only individuals can be a contractor? That wouldn't make much sense, not only because such relationships are rare in schools. Most common are contractors that have been outsourced something like food service. The law would make no sense if it included practically no one. It would mean a company that provides, say, temp staffing within the school, and those temp staffers saw abuse, they too wouldn't be required to report. I have a hard time believing a court would rule the definition to be so narrow. Both the common language understanding of the term and legal literalism would point against that. There's no transitive property here. We're not talking about contractors hired by contractors hired by contractors. We're talking about a contractor and its employees. There is no way for it to exercise this reporting requirement save through it's individual employees.

Re: Flawed Algorithms Are Grading Millions of Students’ Essays

#302
post #183

Earlier quoted context omitted.

Frankly, I can't believe what I am reading. The idea that some "AI" grades essays automatically is idiotic and has nothing to do with education. Where is the place for discussions? Where is the place for ideas confrontation? Where is the place for writing style development? How this AI is supposed to grade things like repetitions (that can be either good rhetorical tool or a mistake, depending on context), etc? Who t…

My guess is that soon or later people will learn how to game that AI. Already been done. http://lesperelman.com/writing-assessment-robo-grading/babel... Here's a sample essay that is complete nonsense and got a perfect score on the GRE. http://lesperelman.com/wp-content/uploads/2015/12/6-6_ScoreI...

Anpther issue is one of copyright: obviously the student is the author. And we all know that the ML scoring subcontractor is keeping copies, with human ratings for later retraining purpouses.

At the time the student takes the test, he should be prompted with the informed choice asking him to grant either 1) no license to keep a copy for training purpouses 2) a non-exclusive license, and the website where he can get a copy of his own essay 3) a public domain license, again with the relevant domain linked so he can find his own and other's essays. 4) any of the above or other as a function of the resulting grade!

At the same time he should also specify his desire for or against attribution, again probably best as a function of the resulting grade. And under what moniker he wishes this contribution to exist.

These options to be filled out during exam time should have no default options (no opt-out), and preferably should be standardized by the community and lobbied for to be mandatorily enforced at state or federal level (forcing examinations to present the student with an informed choice)

A public dataset of legally obtained essays (without scores or names) would already be a very important first step to invite others to make actual performant ML grading systems.

I don't believe the current datasets in these "non-profit" organizations actually comply with copyright law, organizations who don't profit from the grading service towards the state, but do provide a stable ML job on the income from charging the people with financial means to test submissions, enabling a stealth class based society.

Post reply on HN