Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

81–90 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#81
post #26

This does make me wonder if this is susceptible to the same form of trolling as that MS AI got. Commit a load of grossly offensive material to multiple repos, and wait for Copilot to start parroting it. I think they're going to need some human moderation.

Offensive code is the least of my worries. What about vulnerable/exploitable code?

1- re-upload all the shell script you can find, after having inserted `rm -rf --no-preserve-root /` every other line

2- …

3- profit

Re: Copilot regurgitating Quake code, including sweary comments

#82

From the GPLv2 licensed code: https://github.com/id-Software/Quake-III-Arena/blob/master/c... copilot repeats it word for word almost, including comments, and adds an MIT like license up the top

This exact code is all over github, >1k hits

https://github.com/search?q=%22i++%3D+%2A+%28+long+%2A+%29+%...

Re: Copilot regurgitating Quake code, including sweary comments

#83

It seems like a very sensible answer from copilot since the prompt includes "Q_" which makes it obvious that the programmer is specifically looking for the Quake version of this function. To me it doesn't show that copilot will regurgitate existing code when I don't want it to, just that if I ask it to copy some famous existing code for me it will oblige.

I was able to trigger it without the Q prompt before. It just made the a nicer looking gif that way.

I got it to produce more GPL code too, that one is just not entertaining.

Re: Copilot regurgitating Quake code, including sweary comments

#84
post #16

Earlier quoted context omitted.

Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are intro…

While I accept most of the concerns, it's better than your comment suggests. I see some promise for it as a tool for reminding you of a technique or inspiring you to a different approach than you've seen before. For example, I wrote a comment along the lines of "Find the middle point of two 2D positions stored in x, y vectors" and it came up with two totally different approaches in Ruby - one of which I wouldn't have…

Yeah, but... I mean, I guess we all agree that copying code from, let's say StackOverflow without checking if it really does what you want it to do is a bad thing? Now here we have a tool that basically automates that (except it's copying from GitHub, not StackOverflow), and that's supposed to be a good thing? Even if its AI is smarter, you would still have to check the code it suggests, and that can actually be harder than writing it yourself...

Re: Copilot regurgitating Quake code, including sweary comments

#85
post #69

Earlier quoted context omitted.

Why would excluding GPL'd code be enough to not violate licenses? I don't understand why people think MIT or other licenses are free for alls to take code as they wish. The MIT license includes an attribution clause. And, as the linked video shows, Copilot is more than happy to take its code and put your pet license and copyright notice on instead. Isn't that equally as infringing as stealing GPL code? The idea of mi…

Copilot is a tool. If you take copilot's suggestions uncritically and push them to Github, - that's on you.

Yeah, because I always check the code of my programming partner for license violations.

That's more Trainee than Copilot.

Re: Copilot regurgitating Quake code, including sweary comments

#86
post #16

Earlier quoted context omitted.

Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are intro…

Hm odd takes here. It's really weird for software engineers to judge something by its current state and not by its potential state. To me, it's clearly solvable by Copilot filtering the input code by that repository's license. It should only be certain open source licenses, maybe even user-choosable, or code-creators can optionally sublicense their code to Copilot in a very permissable way. Secondly, a way for the cr…

Practically every open source license requires attribution, if copilot has a licensing issue, training a model on only repositories with the same license won't fix it except for the extremely rare licenses which do not require attribution.

Re: Copilot regurgitating Quake code, including sweary comments

#87
post #30

They have 4 hand picked examples on their homepage: https://copilot.github.com/ One has the issue with form encoding: https://news.ycombinator.com/item?id=27697884 The python example is using floats for currency, in an expense tracking context. The golang one uses a word ("value") for a field name that's been a reserved word since SQL-1999. It will work in popular open source SQL databases, but I believe it would bom…

> And these are the hand picked examples. This product seems like it needs some more thought.

Everyone's self-preservation instincts kicking in to attack Copilot is kinda amusing to watch.

Copilot is not supposed to produce excellent code. It's not even supposed to produce final code, period. It produces suggestions to speed you up, and it's on you to weed out stupid shit, which is INEVITABLE.

As a side note, Excel also uses floats for currency, so best practice and real world have a huge gap in-between as usual.

Re: Copilot regurgitating Quake code, including sweary comments

#88
post #5

This is utterly damning. I have already instructed my team that Copilot can never be used for our projects. Compromising the product because of unknowable license demands isn't acceptable in the professional world of software engineering. But if we put the licensing to one side for a moment... 1/ Everything I've seen it generate so far is 'imperative hell'. It is practically a 'boilerplate generator'. That might be u…

I'm not sure we should throw the baby out with the bath water here due to the large blurbs it stubs in when when it doesn't have a lot to go on in mostly empty files. It is a preview release. They are working on proper attribution of suggested code and explainability [1]. Having a stochastic parrot that types faster than I do would be useful in a lot of cases.

Yes, better layers of abstraction could make us more productive in the future, but we're not there yet. By all means, don't accept the larger blurbs it proposes, but there is productivity to be gained in the smaller suggestions. If it correctly intuits the exact rest of the line that you were thinking of, it will save time and not make you lose understanding of the program.

In some areas complete understanding and complete code ownership is required but in a lot of places, it's not. If it produces the work of a moderately skilled developer it would be sufficient. I don't remember all code I write as time passes. If it produces work that I would have produced, then I don't see how that's any different that work that was produced by my past self.

It may feel offensive but a lot of the comments against it sound like rage against the machine/industrialization opponents and the arguments sound pretty similar to those made in the past by those that had their jobs automated away. I'm not sure we're all as unique snowflakes as we like to think we are. Sure, there will be some code that requires an absolute master that is outside the capabilities of this tool. But I'd guess there is a massive amount of code that doesn't need that mastery.

[1] https://docs.github.com/en/github/copilot/research-recitatio...

Re: Copilot regurgitating Quake code, including sweary comments

#90
post #31

I think the problem might be in the training data. Famous code examples are probably copied a lot and therefore appear multiple times in the training data, prompting the neural network to memorise it completely.

Famous code examples are also much more likely to be noticed. For all I know, the thing might be spewing random GPL'd code from the long tail of GitHub all the time and nobody notices because it was written by some random guy and not John Carmack.

Carmack didn't write this: https://en.wikipedia.org/wiki/Fast_inverse_square_root
Post reply on HN