Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Algorithm Didn’t Like My Essay

nytimes.com

Post reply on HN
Print view View original

21–30 of 59 posts

Re: The Algorithm Didn’t Like My Essay

#21

Post by kenrikm »

bermanoid wrote: post #20
majormajor wrote: post #8

Earlier quoted context omitted.

For the standardized test discussed in the article, it says the human graders gave three minutes, on average, to reading each essay. These tests are not wanting art. Writing one of these essays is no more art than writing a basic linked list implementation. It's not concerned with how good a writer a kid is, but that the kid can write at all. Edit: basically, the sheer scale of the grading required for a standardized…

"Art" is not even a question - these tests typically apply to high school kids, and you're lucky if bare-bones basic communication is possible there. Most of these kids are straight-up idiots, and I say this having spent several years teaching them. I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT…

Jaded much? Maybe I'm overly optimistic of the quality of students the school system produces? (I never went to high school, I took the GED and scored in the 100th percentile across the board and went directly to college at 16)

Re: The Algorithm Didn’t Like My Essay

#22

Post by bermanoid »

alextp wrote: post #3

Giving students close access to a program that evaluates essays many times would be the worst possible thing to do. People are generally very good at adapting to systems, and the algorithm would break down pretty quickly as people find out which factors it favors and which ones it penalises (while human graders are arguably less strict in their criteria and can hence recognize these attempts at writing to the letter)…

If students en masse merely ran their shitty essays through spell and grammar check, the quality of even college-level essays would be improved by a huge margin.

For 99% of the high school students I've worked with, their essays would be better if they had to pass Microsoft's spelling and grammar checks before they were submitted. The exceptions were all exceptional enough that they knew that they no longer needed to listen to those automated filters.

Re: The Algorithm Didn’t Like My Essay

#23

Post by unimpressive »

bermanoid wrote: post #20
majormajor wrote: post #8

Earlier quoted context omitted.

For the standardized test discussed in the article, it says the human graders gave three minutes, on average, to reading each essay. These tests are not wanting art. Writing one of these essays is no more art than writing a basic linked list implementation. It's not concerned with how good a writer a kid is, but that the kid can write at all. Edit: basically, the sheer scale of the grading required for a standardized…

"Art" is not even a question - these tests typically apply to high school kids, and you're lucky if bare-bones basic communication is possible there. Most of these kids are straight-up idiots, and I say this having spent several years teaching them. I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT…

>I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT Writing scores.

The best part about that is the fact that people misspell words even with spellcheck! Come on, how hard is it to notice the squiggly lines and correct your error by picking the correct spelling from the drop down list?

And I wouldn't say what I produce is shit, but then I would have said that when I was twelve. And I mostly have to double time on projects and essays, so they probably are shit.

EDIT: And since losethos (Schizophrenic dude who's hellbanned.) asked, the boss; it's pretty clever to use someones tendency for pedantry against them.

Re: The Algorithm Didn’t Like My Essay

#24

Post by adrianN »

InclinedPlane wrote: post #18

We keep paying more and more per student for public education. And what do we get in return? Robo-grading and prison-like treatment of students (closed campuses, zero tolerance taken to ridiculous extent). And again we come around to the notion that if only we'd spend more everything would get better, which has worked so well every time it's happened in the past. The increasing reliance on automated tools and standar…

But this article advocates using automated grading to reduce costs and free up teacher time for better things.

Re: The Algorithm Didn’t Like My Essay

#25

Post by _wp3r »

mingpan wrote: post #6

I think this says more about the assignment and the human grading procedures than anything else. Even the supposedly creative sections in standardized tests are surprisingly narrow in what they consider to be good.

Exactly what I was thinking. The ability to reliably predict human scores, says more about the terrible way we "standardize" creative writing, than about the quality of the algorithms.

If the grading curve of an essay is a checklist how is this finding a surprise?

Re: The Algorithm Didn’t Like My Essay

#26

Post by losethos »

unimpressive wrote: post #23
bermanoid wrote: post #20

Earlier quoted context omitted.

"Art" is not even a question - these tests typically apply to high school kids, and you're lucky if bare-bones basic communication is possible there. Most of these kids are straight-up idiots, and I say this having spent several years teaching them. I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT…

>I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT Writing scores. The best part about that is the fact that people misspell words even with spellcheck! Come on, how hard is it to notice the squiggly lines and correct your error by picking the correct spelling from the drop down list? And I wouldn't…

http://www.noob.us/humor/the-office-dwight-faces-nerd-tortur...

Who's smarter the boss or Dwight?

God says...

49:24 Damascus is waxed feeble, and turneth herself to flee, and fear hath seized on her: anguish and sorrows have taken her, as a woman in travail.

49:25 How is the city of praise not left, the city of my joy! 49:26 Therefore her young men shall fall in her streets, and all the men of war shall be cut off in that day, saith the LORD of hosts.

49:27 And I will kindle a fire in the wall of Damascus, and it shall consume the palaces of Benhadad.

49:28 Concerning Kedar, and concerning the kingdoms of Hazor, which Nebuchadrezzar king of Babylon shall smite, thus saith the LORD; Arise ye, go up to Kedar, and spoil the men of the east.

49:29 Their tents and their flocks shall they take away: they shall take to themselves their curtains, and all their vessels, and their camels; and they shall cry unto them, Fear is on every side.

Re: The Algorithm Didn’t Like My Essay

#27

Post by noonespecial »

InclinedPlane wrote: post #18

We keep paying more and more per student for public education. And what do we get in return? Robo-grading and prison-like treatment of students (closed campuses, zero tolerance taken to ridiculous extent). And again we come around to the notion that if only we'd spend more everything would get better, which has worked so well every time it's happened in the past. The increasing reliance on automated tools and standar…

The problem with "more money" is that the educational system seems to have a sort of hull speed when it comes to adding money.

I'd guess that the same or less is being spent on actual education today and the excesses, whatever they may be, are simply siphoned off by the parasites in the system.

It doesn't seem to matter how much $ per student gets added, teachers still end up buying classroom supplies with their own money. The system just has more $180k "administrators" now than it used to.

Re: The Algorithm Didn’t Like My Essay

#28

Post by btilly »

noonespecial wrote: post #27
InclinedPlane wrote: post #18

We keep paying more and more per student for public education. And what do we get in return? Robo-grading and prison-like treatment of students (closed campuses, zero tolerance taken to ridiculous extent). And again we come around to the notion that if only we'd spend more everything would get better, which has worked so well every time it's happened in the past. The increasing reliance on automated tools and standar…

The problem with "more money" is that the educational system seems to have a sort of hull speed when it comes to adding money. I'd guess that the same or less is being spent on actual education today and the excesses, whatever they may be, are simply siphoned off by the parasites in the system. It doesn't seem to matter how much $ per student gets added, teachers still end up buying classroom supplies with their own…

University education has been going up at 10%/year for decades now. Way above inflation.

It is clear that it is going on an internal dynamic until those dynamics finally conflict with reality. That collision will be..painful. Given the crushing levels of student debt in this country it may happen sooner rather than later.

If I were in charge, I'd institute a simple change. Improve financial aid. Beef up the federal loan guarantees. Only make either available at schools whose tuition was in the bottom 70% when you applied. With that incentive, I think that a lot of schools would find ways to reduce tuitions towards what (inflation adjusted) was more than sufficient several decades ago.

Re: The Algorithm Didn’t Like My Essay

#29

Post by biot »

CapitalistCartr wrote: post #11
biot wrote: post #5

Earlier quoted context omitted.

This inevitably leads to an arms race, with the students and software getting better and better. At some point, sufficiently advanced gaming of essay scores is indistinguishable from students who naturally write well by following the rules which lead to good writing. As long as creativity and originality aren't sacrificed, is it really a problem?

It does not necessarily lead to an arms race. Once the students learn to write to the gradind software, the schools can claim the students are getting better, and point to an 'objective, unbiased' measuring stick. Everyone wins except for society, i.e. sll of us.

I'm assuming there will be competition among the companies who develop such software where, periodically, schools will re-evaluate their provider and select the best one. Company A will demonstrate their competitive advantage by showing how Company B's software incorrectly grades as "Excellent" an essay that looks like English but reads as gibberish.

Company B will correct their software, but meanwhile Company C has come along and introduced a sophisticated analyzer which is able to grade the presence and quality of the logical argument being made in the essay. Company C will demonstrate how Company A and B's software grades as "Excellent" an essay which is correct English and isn't gibberish but makes no logical sense.

After an indeterminate number of iterations with the software getting better and better at finding issues, the only way to game it will be to write a great essay.

Re: The Algorithm Didn’t Like My Essay

#30

Post by dhx »

How well did the algorithms detect insightful analysis, deep understanding beyond the immediate subject matter, factual correctness, salience and an ability to write to a specific audience?

When reading articles on the Internet I look out for superficial factors including:

‣ Misuse of U+0022 typewriter double quotation marks in place of U+201C and U+201D double quotation marks

‣ Misuse of U+002D hyphens in place of U+2013 en dashes and U+2014 em dashes

‣ Poor understanding of comma and semicolon usage

‣ Choice of typeface, use of ligatures and micro-typographical features, amount of leading used…

‣ Misuse of their, there, they’re, where, were, we’re, affect, effect…

‣ RAS syndrome (sīc)

These factors can assist with analysis of writing. An author can generate credibility in certain contexts by using triangular Unicode bullets instead of ASCII asterisks. Use of lengthy and complex words in place of simple English can also be mistaken for credibility. Analysis of these techniques will show that the writer is often attempting to belittle readers. Documentaries utilise similar tactics that the untrained eye will likely mistake for credibility. Soft filter effects, use of bookcase backdrops for expert interviews and silence have profound manipulative effects on viewers.

Grading algorithms will likely reward use of manipulative and belittling writing techniques and penalise honest superficial mistakes. I would rather read the honest opinion from an author that misuses their and there than have to carefully analyse the carefully crafted writings of a dishonest linguist.

As an aside, I recommend bookmarking The Browser[1] (Writing Worth Reading). It would be interesting to see the results of algorithmic essay grading against this curated collection of articles from across the Internet.

[1] http://thebrowser.com/

Post reply on HN