Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It Wasn't AI

itwasntai.com

Post reply on HN
Print view View original

61–70 of 134 posts

Re: It Wasn't AI

#61

Post by rorroe53 »

simonw wrote: post #5

On the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil. Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable. The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the d…

But why should educational instutions care? Education is a business, students are the customers (or in countries with state-funded education, the government). If AI helps people graduate faster, that's more money to the institutions, less effort to the students, and nice statistics for the governments.

At least in my country most degrees aren't worth much anyway, they just open you doors to internships where you really learn stuff. AI isn't going to make the situation any worse.

Re: It Wasn't AI

#62

Post by scythe »

tejtm wrote: post #30
aripickar wrote: post #12

IMO one of the way that most schools are going to end up being able to detect plagiarism is going to be a custom word processor (or something similar) that can track all of the edits made into a document. Basically, have the students type an essay where all of the keystrokes are recorded by the program, and so it can be detected by the program whether someone is copy and pasting whole essays, or if someone is actuall…

Yet another arms race. Use this key logging training dataset to generate a simulated realtime response on the usb port.

LLMs are useful for a variety of things. What you're describing would only be useful for students cheating on assignments. I doubt that it will attract the many millions of dollars spent on training GPT-4.

But more importantly, LLMs are always available over the Internet. If students need to use a physical device to cheat, that's already a big step forward, since it increases the chance of detection — a key factor in deterring misbehavior.

Re: It Wasn't AI

#63

Post by jfghi »

Ironically, the best detector for plagiarism would be a 15 minute conversation asking the student about their research and opinions on the topics written, kind of like interviewing someone who claims redis expertise on their resume.

Re: It Wasn't AI

#64

Post by simonw »

rorroe53 wrote: post #61
simonw wrote: post #5

On the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil. Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable. The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the d…

But why should educational instutions care? Education is a business, students are the customers (or in countries with state-funded education, the government). If AI helps people graduate faster, that's more money to the institutions, less effort to the students, and nice statistics for the governments. At least in my country most degrees aren't worth much anyway, they just open you doors to internships where you real…

They care because this is massively disruptive to the way they teach at the moment. They have decades of practices in place for how they evaluate students which don't work if students can have AI do the work for them.

They can chose to reinvent everything about how they operate, or they can pay money to a company that promises to make that problem go away for them.

It's not surprising that many of them are trying the latter option first.

Re: It Wasn't AI

#65

Post by numbsafari »

jandrese wrote: post #25
aripickar wrote: post #12

IMO one of the way that most schools are going to end up being able to detect plagiarism is going to be a custom word processor (or something similar) that can track all of the edits made into a document. Basically, have the students type an essay where all of the keystrokes are recorded by the program, and so it can be detected by the program whether someone is copy and pasting whole essays, or if someone is actuall…

When I was in college we had a number of group projects and I thought the whole time that it would make a ton of sense for the professor to set up a class repo (I'm a old person so they would be a CVS repo at the time) and be able to see exactly what each person had contributed to the project. Even for single person projects it would have made it so much easier to detect cheaters. I also think it might light a fire u…

What % of the grade should be based on LOC and what % based on story points?

Re: It Wasn't AI

#66

Post by jandrese »

function_seven wrote: post #53
jandrese wrote: post #32

Earlier quoted context omitted.

So if you submitted the US Constitution word for word for your assignment you wouldn't expect the cheating detector to flag it? How isn't this flagrant plagiarism?

I would expect a plagiarism detector to flag it. I would not expect an "AI wrote this" to flag it, because it was not written by AI.

An AI detector is basically a plagiarism detector trained on AI datasets. Fundamentally it's about discovering writing that is too similar to existing writing, which is why I'm concerned about false positives.

Re: It Wasn't AI

#67

Post by jandrese »

numbsafari wrote: post #65
jandrese wrote: post #25

Earlier quoted context omitted.

When I was in college we had a number of group projects and I thought the whole time that it would make a ton of sense for the professor to set up a class repo (I'm a old person so they would be a CVS repo at the time) and be able to see exactly what each person had contributed to the project. Even for single person projects it would have made it so much easier to detect cheaters. I also think it might light a fire u…

What % of the grade should be based on LOC and what % based on story points?

Typically I'd expect the group project to be graded on its own and all students get the same grade from the project. However, when a project shows that some of the participants committed zero lines of code or suddenly dropped in enormous blocks then they should be asked about it. At the very least they should be encouraged to use branches and make frequent commits like in the real world.

Re: It Wasn't AI

#68

Post by valine »

jandrese wrote: post #51
valine wrote: post #33

Earlier quoted context omitted.

I think you overestimate the competence of the majority of professors. They can’t require version control if they don’t understand what it is or how to use it.

Back in the 90s I could kind of see this angle, but today it's so easy to set up a Gitlab there is no excuse.

The problem isn't the accessibility of Git. I agree that it's easy enough to set up a Github account today.

I've been somewhat of a Git evangelist. I've tried and failed countless times to convince people of the utility of version control. Perhaps I'm just a poor teacher, but in my experience, the features that make version control useful are too esoteric for most people to grasp.

This may come off as arrogant and jaded, but I would speculate that at least 50% of the population is incapable of learning Git without extensive coaching. That's not to say it couldn’t be useful for most people; it's just that they can’t envision Git’s utility for themselves.

Utilizing version control to combat AI generated papers would require students and teachers have a deep enough understand of git to break their work up into small commits and branches. I don’t see that happening outside of the CS departments of big 10 schools.

Re: It Wasn't AI

#69

Post by guy98238710 »

It's not just university students anymore. My 11yo kid got accused of being a robot on a physics competition where the only reward is a (paid) summer camp full of extra physics lessons. All that was needed to trigger the accusation was a bit less fluent explanation of the solution, something you would expect from a student struggling with a difficult task. People are growing unreasonably paranoid.

Re: It Wasn't AI

#70

Post by r_hoods_ghost »

jfghi wrote: post #63

Ironically, the best detector for plagiarism would be a 15 minute conversation asking the student about their research and opinions on the topics written, kind of like interviewing someone who claims redis expertise on their resume.

This is essentially the Oxbridge tutorial system where you have an hour(ish) meeting once(ish) a week with one or two other students and your tutor and talk over what you've learned, are set an assignment or three and have to answer any questions about the last week's assignments. A slightly more scalable version is the seminar system where you have up to 10 students and a tutor and you do roughly the same thing. It only works if participation is mandatory, missing seminars / tutorials is penalised and tutors are given leeway to adjust grades based on semina performance or flag students who do really well on their essays but appear incapable of explaining or defending what they've written.
Post reply on HN