Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A.I. and the Future of Cheating

onezero.medium.com

Post reply on HN
Print view View original

31–40 of 50 posts

Re: A.I. and the Future of Cheating

#31

Post by jacobwilliamroy »

RcouF1uZ4gsC wrote: post #3

> What happens when universities can’t tell whether an essay is written by a human or an algorithm? Then in that scenario, that means that essays are worthless at evaluating students, and if AI essays really are as good as human essays, then teaching the writing of essays at the college level is a waste of time. However, looking at the generated essay, it seems to me that the real problem is that universities have re…

I had a teacher who did pass-fail essays. His rubric only had one box "Did I feel anything while reading this essay?" I think the technology still needs to develop much more before it can fool Mr. Thayer's grading system.

Re: A.I. and the Future of Cheating

#32

Post by virmundi »

symplee wrote: post #16

Highly recommend testing this new cheating tool out by entering a custom prompt at https://talktotransformer.com/ (which uses GPT-2) and reading what the AI generates for your custom essay snippet. A single sentence with a euphemism should suffice... "I" was able to create some hilarious franken-stories. Anyone care to share their greatest hits?

That was interesting, but it is no Crashtest Dummies.

Prompt: Once there was this kid who got into an accident and couldn't come to school.

Essay: The police told me they would call me if they found out if I would come. And I couldn't wait.

I knew how to talk to police because I had done that all of my life. But this guy was different. I talked to the cops and told them that I wasn't going to tell on him and they were cool with that. They let me off easy.

I wasn't going to come on to him, because I knew if he found out that I'd call him the next day I would probably be fired, so it was just a waste of time.

The next morning the police called me, and I had to sit there for five hours. I was really nervous and the whole time I was thinking about how I'd never really seen this kid in my life. And then after they told me that I was on the payroll I was really excited and I kept thinking that if he called me I would have to tell him that I'd never been fired before.

Then when I was going over the names in his file

Re: A.I. and the Future of Cheating

#33

Post by woliveirajr »

pmoriarty wrote: post #9
RcouF1uZ4gsC wrote: post #3

> What happens when universities can’t tell whether an essay is written by a human or an algorithm? Then in that scenario, that means that essays are worthless at evaluating students, and if AI essays really are as good as human essays, then teaching the writing of essays at the college level is a waste of time. However, looking at the generated essay, it seems to me that the real problem is that universities have re…

Students could just be required to write the essays in class rather than as take-home assignments. Also, if there's a great discrepancy in style, skill, or content of in-class essays vs take-home essays, then that could be considered evidence of cheating.

Stylometry is a hard task to be performed by humans and by computers... I've studied it for a while (master degree and PhD), and while we could have some success (>80% in verifying the authorship of a document against 100 authors), I think that it would be hard to make judgements solely based on a percentage below 99.9999%.

OTOH, perhaps I should begin to study this matter again :-)

Re: A.I. and the Future of Cheating

#34

Post by comicjk »

RcouF1uZ4gsC wrote: post #3

> What happens when universities can’t tell whether an essay is written by a human or an algorithm? Then in that scenario, that means that essays are worthless at evaluating students, and if AI essays really are as good as human essays, then teaching the writing of essays at the college level is a waste of time. However, looking at the generated essay, it seems to me that the real problem is that universities have re…

Computers have been able to do arithmetic faster, better, and cheaper than humans for decades. But we still teach children to do arithmetic, and grade them on how well they do it (almost never as well as a computer). Why? Because doing more useful and creative math is a lot easier if you can do a little arithmetic.

The purpose of college essays has never been the quality of the product. The purpose is to have students practice so that, when they really do have to convey something in writing, they can convey it well. If an idea exists only in my head, it will take a very long time for an essay-writing bot to write it for me.

Re: A.I. and the Future of Cheating

#36

Post by g82918 »

analog31 wrote: post #28
benbreen wrote: post #20

Earlier quoted context omitted.

Am a university professor who frequently assigns essays, so this is something I've been thinking a lot about. The problem is not that AI can successfully write a college-level essay (maybe someday, but certainly GPT-2 isn't even close to that level yet). It's that something along the lines of GPT-2 can emulate the superficial appearance of an essay. The actual facts, quotes, and references it generates are generally…

>>> Unfortunately, a lot of essays in university lecture courses are graded by underpaid TAs who are skimming quickly, and who might be inclined to give an AI-written essay a D or a C- for effort rather than recognizing it as fraud. This raises an interesting point, which is that there seems to be a relationship between essay-writing and essay-reading. The students are able to cheat on essay-writing because the schoo…

>This raises an interesting point, which is that there seems to be a relationship between essay-writing and essay-reading. The students are able to cheat on essay-writing because the school is already cheating on essay-reading.

When they pay so little why would a person spend hours reading and analyzing one essay. The bigger issue if the lack of resources devoted to grading and not that the people on the ground are cheating.

edit: I think the commentator's emphasis should be on the lack of resources and not a supposed negligence on the grader's part.

Re: A.I. and the Future of Cheating

#37

Post by Aperocky »

comicjk wrote: post #34
RcouF1uZ4gsC wrote: post #3

> What happens when universities can’t tell whether an essay is written by a human or an algorithm? Then in that scenario, that means that essays are worthless at evaluating students, and if AI essays really are as good as human essays, then teaching the writing of essays at the college level is a waste of time. However, looking at the generated essay, it seems to me that the real problem is that universities have re…

Computers have been able to do arithmetic faster, better, and cheaper than humans for decades. But we still teach children to do arithmetic, and grade them on how well they do it (almost never as well as a computer). Why? Because doing more useful and creative math is a lot easier if you can do a little arithmetic. The purpose of college essays has never been the quality of the product. The purpose is to have student…

Speaking from experience, the person that writes best code (in terms of both raw quality and readability) also seem to write the best docs.

Maybe teaching essays are sideways to efficient communication?

Re: A.I. and the Future of Cheating

#38

Post by cellular »

benbreen wrote: post #20
notduncansmith wrote: post #13

Earlier quoted context omitted.

This doesn’t address the first point of the parent, which is that if AI can pass the essay Turing test, then perhaps college students shouldn’t be paying thousands of dollars and spending hundreds of hours learning to produce essays? If essays are a proxy for some other process, perhaps we should find a better proxy.

Am a university professor who frequently assigns essays, so this is something I've been thinking a lot about. The problem is not that AI can successfully write a college-level essay (maybe someday, but certainly GPT-2 isn't even close to that level yet). It's that something along the lines of GPT-2 can emulate the superficial appearance of an essay. The actual facts, quotes, and references it generates are generally…

I frequently wonder why the cost of college is rising while the teaching / grading is relegated to teacher assistants, and now, AI.

Re: A.I. and the Future of Cheating

#39

Post by mikorym »

Aperocky wrote: post #37
comicjk wrote: post #34

Earlier quoted context omitted.

Computers have been able to do arithmetic faster, better, and cheaper than humans for decades. But we still teach children to do arithmetic, and grade them on how well they do it (almost never as well as a computer). Why? Because doing more useful and creative math is a lot easier if you can do a little arithmetic. The purpose of college essays has never been the quality of the product. The purpose is to have student…

Speaking from experience, the person that writes best code (in terms of both raw quality and readability) also seem to write the best docs. Maybe teaching essays are sideways to efficient communication?

Although we have machines that now cut and bind books, the best book binding (and most aesthetic) are done by hand by enthusiasts on, e.g., Reddit. Moreover, such book binders don't feel threatened by machines, I would think, like how people who do DIY do it because they like it, rather than because they have to do it.

Even if an AI can write an essay, that would not stop me from writing. Essays have another important dimension, which is that of creativity and the message. I think the only real relevance is in the context of school work, outside of that I don't think the AI's ability is that important. People are intelligent already, and competition between writers or the general tendency to influence each other sometimes is even a good thing.

Re: A.I. and the Future of Cheating

#40

Post by ses1984 »

benbreen wrote: post #20
notduncansmith wrote: post #13

Earlier quoted context omitted.

This doesn’t address the first point of the parent, which is that if AI can pass the essay Turing test, then perhaps college students shouldn’t be paying thousands of dollars and spending hundreds of hours learning to produce essays? If essays are a proxy for some other process, perhaps we should find a better proxy.

Am a university professor who frequently assigns essays, so this is something I've been thinking a lot about. The problem is not that AI can successfully write a college-level essay (maybe someday, but certainly GPT-2 isn't even close to that level yet). It's that something along the lines of GPT-2 can emulate the superficial appearance of an essay. The actual facts, quotes, and references it generates are generally…

Imagine a model trained just on college essays, and you feed it an outline. A sentence or two intro, some topic sentences, and citations.

Probably if someone had a dataset of essays large enough, you could do this right now and cause a lot of trouble.

Post reply on HN