Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JPlag – Detecting Software Plagiarism

github.com

Post reply on HN
Print view View original

61–70 of 85 posts

Re: JPlag – Detecting Software Plagiarism

#61

Post by hooverd »

fn-mote wrote: post #60
hooverd wrote: post #9

It's funny how we drill the idea that everything must be reimplemented from first principles into students, only to flip that when they join the workforce.

> It's funny how we drill the idea that everything must be reimplemented from first principles into students, only to flip that when they join the workforce. It is funny , but that doesn’t mean it’s wrong. PS If the domain in your profile is supposed to be valid … it isn’t.

Yea, it's a great way to learn.

Thanks, I need to redo my website/blog/whatever. I'm still getting email at it!

Re: JPlag – Detecting Software Plagiarism

#62

Post by omoikane »

emeryberger wrote: post #6

JPlag, like similar plagiarism detectors, is vulnerable to attack. We outline the attack in this paper and show its effectiveness against JPlag and another widely used plagiarism detector, Moss. Note that this was written in 2020, in the pre “CheatGPT” era! https://arxiv.org/abs/2010.01700 Mossad: Defeating Software Plagiarism Detection Breanna Devore-McDonald, Emery D. Berger Automatic software plagiarism detection…

The usual defense against these is to ask students to explain their submitted work. Randomly generated dead code would likely be even more difficult for the students to explain.

Though a counterargument to this would be that teachers don't have time to interview every student. If Mossad is so good that teachers can't pick out the objectively suspicious subset, they might need to subjectively pick a random sample with varying amount of personal bias involved.

Re: JPlag – Detecting Software Plagiarism

#63

Post by immibis »

ActualTeacher wrote: post #35
hooverd wrote: post #9

It's funny how we drill the idea that everything must be reimplemented from first principles into students, only to flip that when they join the workforce.

> only to flip that when they join the workforce. A surprisingly large number of people do not realise that code on StackOverflow is under a relatively restrictive license. https://meta.stackexchange.com/questions/12527/do-i-have-to-... Some companies take this very seriously and others do not care at all. And, of course, there are companies that outright ban any library without the right licence.

Most companies realize they can do what they want as long as there are no actual consequences. These companies prosper under the rules of natural selection, compared to the companies which are afraid of doing things because some paper says they shouldn't do it even though nothing will happen if they do it.

Re: JPlag – Detecting Software Plagiarism

#64

Post by hooverd »

dspillett wrote: post #26
hooverd wrote: post #9

It's funny how we drill the idea that everything must be reimplemented from first principles into students, only to flip that when they join the workforce.

Building things from first principles is a great way to instil understanding about how things work and why, which helps the understanding of problems later. When doing things for real in the workforce, reinventing core parts yourself is often not the best way, if only because you'll reinvent already fixed bugs too or that it wastes time, and that should be explained too. But understanding how things work below the ou…

Yea, you should learn how things work. I'm just worried that overly clever plagiarism detection would ding you for things like converging on common design patterns.

Re: JPlag – Detecting Software Plagiarism

#65

Post by Johnbot »

diggan wrote: post #57
michaelmior wrote: post #53

Earlier quoted context omitted.

It would certainly depend to some degree on the complexity of the assignment. But it's also not that unusual for legitimate, non-plagiarized submissions to have dead code.

Sure, but is it not unusual to have "dead code consisting of lines from the original file repeated in random locations"? That would certainly stick out in any other environment (like a professional one). I didn't study anything related to computers/software/programming in school, so I don't know what level is expected. But if I was tutoring someone and they handed me something with dead code in random locations in it…

I think two things are at play here.

1. Students will frequently just try things until it works, move code around, etc., leading to very messy code. 2. Graders often do not look at individual assignments unless there is a reason to do so, often relying on automated test suites. And when they do look, I'd bet their first reaction is something like "I don't know why they're repeating themselves like this, but my rubric only penalizes them for 5 points here..."

Re: JPlag – Detecting Software Plagiarism

#66

Post by meiraleal »

thiht wrote: post #55

I was actually looking for something like this a few days ago! There’s an open source tool which I love the idea of (basically a tool for declarative integration tests), but I really don’t like it’s implementation. I tried to contribute to improve it, but it’s too much work and it will never fit my ideal. So I basically decided to "redo it but better", and I’m also tempted to make it a paid, proprietary tool because…

> to make sure I didn’t accidentally plagiarize open source code.

If you didn't plagiarize, you don't need to run the tool. If you did plagiarize and want to hide it, tho...

Re: JPlag – Detecting Software Plagiarism

#67

Post by jrm4 »

omoikane wrote: post #62
emeryberger wrote: post #6

JPlag, like similar plagiarism detectors, is vulnerable to attack. We outline the attack in this paper and show its effectiveness against JPlag and another widely used plagiarism detector, Moss. Note that this was written in 2020, in the pre “CheatGPT” era! https://arxiv.org/abs/2010.01700 Mossad: Defeating Software Plagiarism Detection Breanna Devore-McDonald, Emery D. Berger Automatic software plagiarism detection…

The usual defense against these is to ask students to explain their submitted work. Randomly generated dead code would likely be even more difficult for the students to explain. Though a counterargument to this would be that teachers don't have time to interview every student. If Mossad is so good that teachers can't pick out the objectively suspicious subset, they might need to subjectively pick a random sample with…

Yup. I sort of independently discovered this mechanism, and not just for "cheating," but for group work. Didn't even have to go individual interviews.

It was simple, I let students work in groups to do coding stuff (it's an intro type of class with students of varying skill levels). I had them work on a project together all they wanted, letting them know that it would be turned in about a month or so before the end of the semester. I would review them and then, in class, they would INDIVIDUALLY be quizzed on their own teams project; down to e.g.

"You have a function blahblah, explain what it does. What would happen if I passed it X?"

Forces them to work together and sort of study together. Kind of puts a bit more pressure on the less knowledgable, but probably worth it.

Re: JPlag – Detecting Software Plagiarism

#68

Post by westurner »

Should a plagiarism score be considered when generating code with an infinite monkeys algorithm with selection or better?

Would that result in inability to write code in a clean room, even; because eventually all possible code strings and mutations thereof would already be patented.

For example, are three notes or chords copyrightable?

Re: JPlag – Detecting Software Plagiarism

#69

Post by teruakohatu »

Johnbot wrote: post #50
diggan wrote: post #18

Earlier quoted context omitted.

> A user study shows that graduate student assistants consistently rate Mossad-generated code as just as readable as authentic student code. Do you have any small examples on a program that was transformed/generated with Mossad that we could compare against the original? As far as I can tell, the paper just have a really tiny example function.

An example of a Mossad generated file would be the source file plus a bunch of dead code. The dead code consists of lines from the original file repeated in random locations (plus, if you are using an "entropy file", random lines of code that were successful mutations from previous generations of Mossad). As it turns out, a lot of student code can look this way anyway. Something crazy like 70% of authentic student co…

> As it turns out, a lot of student code can look this way anyway. Something crazy like 70% of authentic student code can have dead code in assignment submissions.

Having assessed student code this does not surprise me. Source code control late at night for students, especially non-CS majors, tends to be variations of "append a number to the end of the function name" eg. sum1(x, y) sum2(x, y) ... sumTHISREALLYWORKS(x, y).

That said, if dead code was being used to hide plagiarism, which is something I had not considered before, then telling students they would be marked down for dead code would probably be enough to stop it.

Re: JPlag – Detecting Software Plagiarism

#70

Post by nsajko »

ziddoap wrote: post #40
ActualTeacher wrote: post #38

Earlier quoted context omitted.

The output of plagiarism tools should only serve as a hint to look at a pair of solutions more closely. All judgement should be derived entirely from similarities between solutions and not some artificial similarity score computed by some program.

Unfortunately, this is not really what happens in my experience. The output of plagiarism tools is taken as fact (especially at high school levels). Without extraordinary evidence of the tool being incorrect, students have no recourse, even if they could sit and explain the thought process behind every word/line of code/whatever.

Lousy high school.
Post reply on HN