Bringing SEO mentality to standardized testing what could go wrong.
Flawed Algorithms Are Grading Millions of Students’ Essays
31–40 of 302 posts
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#32It seems like these accumulated errors in the educational system and filters needed to get through it would create a market inefficiency that could be exploited by a firm willing to ignore degrees, grades, and test scores and judge for themselves whether a candidate can do the job they're being hired for.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#33Unlike a multiple choice test where the primary audience is automated graders, the primary audience for an essay is other humans. If even Google and Facebook with their billions of dollars and billions of posts worth of data, still cannot always understand the intent and purpose of written content, what hope do these algorithms have? If it is cost-prohibitive for every essay to be graded by humans, then they should b…
If it is cost-prohibitive then then maybe we should adjust the economic model, not abandon the measurement.
I mean that's what the automated grading systems are trying to do but it seems like people don't like them very much.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#34My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.) One question she had to grade was essentially, "What's something you want your teacher to know about you?" It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.) She sa…
Some of these will be 100% true as well. But don't make the mistake that there are no kids who go for shock value or are wantonly manipulative when they know it can't come back to them.
So how many are true and how many false? I have no clue. Literally none. And no it doesn't make me feel any better about the screams of existential agony even if that were a low percentage. Could be high too.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#35To me this brings up the absurdity of having essays on standardized tests. What about an essay is standardized? It's a totally nonsensical premise.
This always gets made into some kind of techluminati conspiracy for the machines to ingrain structural racism whereas it's pretty clear all the algorithms fail to do is improve an already bad situation stemming from a flawed premise.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#36Earlier quoted context omitted.
> If it is cost-prohibitive for every essay to be graded by humans, then they should be dropped from the tests. Apparently it is. But everyone still wants writing to be assessed…
The SAT dropped the writing section a few years ago, and many schools don't care about the GRE writing score.
They did drop the writing but replaced it with an optional essay. Most upper tier schools require applicants to take the essay.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#37I imagine that any student that experimented with the form of the essay or wrote an exceptionally well argued piece in simple language would not have their test graded appropriately either. Any essay writing test which could be adequately graded by a machine is not testing anything of value. Edit: I’ll further add that as soon as people’s careers depend on a metric, the metric becomes useless as a metric, because it…
Almost nobody involved is incentivized to accurately measure student’s writing ability It's the same reason you see keyword posters in math education. "Together" means "plus", that kind of thing. It's completely worthless, except for one-step problems, and even then it doesn't always work. What is happening is collusion between teachers and testmakers. You can't teach understanding, but you can teach test-passing tec…
If I have 3 left shoes colored blue green and red, and you have 2 right shoes colored black and white, how many pairs can we make if our lefts and rights are put together?
Hint: together does not mean plus.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#38My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.) One question she had to grade was essentially, "What's something you want your teacher to know about you?" It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.) She sa…
Or...report it to the police? I’d gladly risk my job to do the right thing in that instance.
Hand-graded standardized tests are usually anonymized.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#39My mother worked grading standardized tests. It was a hellish job for many reasons (limited breaks, etc.) One question she had to grade was essentially, "What's something you want your teacher to know about you?" It was an essay answer, and she was supposed to grade it on grammar, etc. Just the mechanical aspects of writing. (The real question explained the details more, but that was the core of the question.) She sa…
Or...report it to the police? I’d gladly risk my job to do the right thing in that instance.
What my mom saw had an ID number on it. No other demographics. And she was grading from multiple states.
So do what? Contact her local police?
With a written accusation from a child? Is that enough to get a warrant to force the company to release the demographic information?
And people don't work at a job like that because they want to. They work there because they need the money.
Everything she took in and out of there was monitored, too. So it's not like she can go to the Xerox, and walk out of there with a copy.
It's beyond dehumanizing. For everyone. The kid, the people who work there.
Re: Flawed Algorithms Are Grading Millions of Students’ Essays
#40From the article: "All essays scored by E-rater are also graded by a human and discrepancies are sent to a second human for a final grade. Because of that system, ETS does not believe any students have been adversely affected by the bias detected in E-rater."