Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JPlag – Detecting Software Plagiarism

github.com

Post reply on HN
Print view View original

71–80 of 85 posts

Re: JPlag – Detecting Software Plagiarism

#71

Post by norswap »

emeryberger wrote: post #6

JPlag, like similar plagiarism detectors, is vulnerable to attack. We outline the attack in this paper and show its effectiveness against JPlag and another widely used plagiarism detector, Moss. Note that this was written in 2020, in the pre “CheatGPT” era! https://arxiv.org/abs/2010.01700 Mossad: Defeating Software Plagiarism Detection Breanna Devore-McDonald, Emery D. Berger Automatic software plagiarism detection…

As someone that actually used JPlag as a university TA, I think if the students are smart enough to implement this, they're probably smart enough to do whatever assignment we've asked of them (unless there's a easy peasy program to do the transformation, but I don't think it's the case here).

The usage of the tool is basically a deterrent against a very low-hanging cheating fruit for students (some still tried and thought changing the variable names would help them...)

Re: JPlag – Detecting Software Plagiarism

#72

Post by thiht »

meiraleal wrote: post #66
thiht wrote: post #55

I was actually looking for something like this a few days ago! There’s an open source tool which I love the idea of (basically a tool for declarative integration tests), but I really don’t like it’s implementation. I tried to contribute to improve it, but it’s too much work and it will never fit my ideal. So I basically decided to "redo it but better", and I’m also tempted to make it a paid, proprietary tool because…

> to make sure I didn’t accidentally plagiarize open source code. If you didn't plagiarize, you don't need to run the tool. If you did plagiarize and want to hide it, tho...

There’s big sections of code I wrote in the original open source lib. I didn’t copy paste the code but the implementation in this component is obviously pretty close. I’m the copyright holder of this code anyway so it should not be an issue, but I’d rather not take the risk.

Plagiarism is not always clear cut because life is messy. That’s why Wine doesn’t allow contributions from people who have seen Windows source code[1] for instance, even though it could be good faith contributions with experience instead of plagiarism

[1]: https://wiki.winehq.org/Developer_FAQ#Who_can't_contribute_t...?

Re: JPlag – Detecting Software Plagiarism

#73

Post by maleldil »

playingalong wrote: post #3

Can I use it to detect Copy & Paste within my company's own codebase?

I've had some success with PMD/CMD [1]. It's Java-based, though, which could be a pain, depending on your setup. If your codebase uses Python, you can use Pylint for this, too.

[1] https://pmd.github.io/pmd/pmd_userdocs_cpd.html

Re: JPlag – Detecting Software Plagiarism

#74

Post by withinboredom »

teruakohatu wrote: post #69
Johnbot wrote: post #50

Earlier quoted context omitted.

An example of a Mossad generated file would be the source file plus a bunch of dead code. The dead code consists of lines from the original file repeated in random locations (plus, if you are using an "entropy file", random lines of code that were successful mutations from previous generations of Mossad). As it turns out, a lot of student code can look this way anyway. Something crazy like 70% of authentic student co…

> As it turns out, a lot of student code can look this way anyway. Something crazy like 70% of authentic student code can have dead code in assignment submissions. Having assessed student code this does not surprise me. Source code control late at night for students, especially non-CS majors, tends to be variations of "append a number to the end of the function name" eg. sum1(x, y) sum2(x, y) ... sumTHISREALLYWORKS(x…

I mean. Should be doing that anyway. Code doesn’t just exist for the computer, but also for humans who have to maintain it.

Re: JPlag – Detecting Software Plagiarism

#75

Post by willy_k »

playingalong wrote: post #3

Can I use it to detect Copy & Paste within my company's own codebase?

What do you need that for? It’s not really cheating when it’s going into a product, if it works it works. In a hypothetical ideal corporate environment wouldn’t it be preferred if one could save company time by copy and pasting?

Re: JPlag – Detecting Software Plagiarism

#76

Post by teruakohatu »

withinboredom wrote: post #74
teruakohatu wrote: post #69

Earlier quoted context omitted.

> As it turns out, a lot of student code can look this way anyway. Something crazy like 70% of authentic student code can have dead code in assignment submissions. Having assessed student code this does not surprise me. Source code control late at night for students, especially non-CS majors, tends to be variations of "append a number to the end of the function name" eg. sum1(x, y) sum2(x, y) ... sumTHISREALLYWORKS(x…

I mean. Should be doing that anyway. Code doesn’t just exist for the computer, but also for humans who have to maintain it.

> I mean. Should be doing that anyway. Code doesn’t just exist for the computer, but also for humans who have to maintain it.

Harsh! I like to think I am good lecturer.

Depends on the specification of the assignment. In my case I teach data science not software development so the specification is not "bullet proof code that won't break when pytorch releases a new version tomorrow" but rather statistical and data rigour. This is where spent my time when marking, not how maintainable the code is.

CS students turn in MUCH better code, but frequently data is leaking into tests or validation sets etc. making the results either meaningless or compromised.

At the end of the day code quality is strongly correlated to grades.

Re: JPlag – Detecting Software Plagiarism

#77

Post by zelphirkalt »

emeryberger wrote: post #6

JPlag, like similar plagiarism detectors, is vulnerable to attack. We outline the attack in this paper and show its effectiveness against JPlag and another widely used plagiarism detector, Moss. Note that this was written in 2020, in the pre “CheatGPT” era! https://arxiv.org/abs/2010.01700 Mossad: Defeating Software Plagiarism Detection Breanna Devore-McDonald, Emery D. Berger Automatic software plagiarism detection…

In an educational setting the plagiarism tools are probably most wanted by lecturers, but least useful. Do they teach every individual differently? If not, then there is not much surprise, if elementary ideas are expressed in very similar ways. So some cases of very similar solutions are bound to happen, hopefully not throwing shadow without proof of plagiarism.

Re: JPlag – Detecting Software Plagiarism

#78

Post by serf »

pdntspa wrote: post #58
hatmatrix wrote: post #29

Earlier quoted context omitted.

Surprising because you get different answers each time you ask ChatGPT.

ChatGPT answers don't differ that much without being prompted to do so

yeah but the prompt itself generally adds sufficient randomness to avoid the same verbatim answer each time.

as an example just go ask it to write any sufficiently average function. use different names and phrases for what the function should do; you'll generally get a different flavor of answer each time, even if the functions all output the same thing.

sometimes the prompt even forces the thing to output the most naive implementation possible due to the ordering or perceived priority of things within the requesting prompt.

it's fun to use as a tool to nudge it into what you want once you get the hang of the preconceptions it falls into.

Re: JPlag – Detecting Software Plagiarism

#79

Post by withinboredom »

teruakohatu wrote: post #76
withinboredom wrote: post #74

Earlier quoted context omitted.

I mean. Should be doing that anyway. Code doesn’t just exist for the computer, but also for humans who have to maintain it.

> I mean. Should be doing that anyway. Code doesn’t just exist for the computer, but also for humans who have to maintain it. Harsh! I like to think I am good lecturer. Depends on the specification of the assignment. In my case I teach data science not software development so the specification is not "bullet proof code that won't break when pytorch releases a new version tomorrow" but rather statistical and data rigo…

That seems like readability is even more important! I've taught programming to friends and family my entire life (to anyone who wants to learn), and one thing I always focus on is 'telling a story with comments', explaining how, where, and why data flows through the code. At the end, reread your comments and your code and figure out which one is wrong; then refactor.

Re: JPlag – Detecting Software Plagiarism

#80

Post by emeryberger »

franga2000 wrote: post #8
jaimex2 wrote: post #4

Earlier quoted context omitted.

Kinda rare these days with ChatGPT

MOSS seems to be pretty good finding multiple people using LLM-generated code and flagging them as copies of each other. I imagine it would also be a good idea to throw the assignment text into the few most popular LLMs and feed that in as well, but I don't know of anyone who has tried this.

FWIW the attack we describe in the paper works against MOSS, too (that was the original inspiration for the name, “Mossad”).
Post reply on HN