Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

51–60 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#51

Post by NumberCruncher »

michaelt wrote: post #40
sydthrowaway wrote: post #10

What causes this in a net? I’m guessing the RNN gets in a catastrophic state..

This is the network working as designed. I mean, if you wrote an autocomplete system for written english and asked it to complete the sentence "O Romeo, Romeo" what would you expect to happen? You'd expect it to complete to "O Romeo, Romeo, wherefore art thou Romeo?" - a very famous quote. How else could you produce the single right output for that unique input, other than memorising and regurgitating?

> You'd expect it to complete to "O Romeo, Romeo, wherefore art thou Romeo?" - a very famous quote.

What about completing it to "O Romeo, Romeo, brave Mercutio is dead", based on the context, as advertised?

Re: Copilot regurgitating Quake code, including sweary comments

#52

Post by maccard »

saurik wrote: post #39
bohemian99 wrote: post #32

Earlier quoted context omitted.

My question is would Copilot be useful if you could choose the codebase it would be drawing from? Almost as an internal company tool?

If you find yourself copying code someone else in your organization wrote rather than abstracting it to a function in a shared library or building a more declarative framework to manage the problem, something horrible has happened.

Sometimes boilerplate is unavoidable. As an example, how do you send a GET request with libcurl in C with an authorization header? I can't tell you offhand, but I can tell you the file in my codebase that does have it, because I've duplicated the logic for two separate systems.

Re: Copilot regurgitating Quake code, including sweary comments

#53

Post by abecedarius »

0-_-0 wrote: post #29
0-_-0 wrote: post #17

This is a very famous function [0] and likely appears multiple times in the training set (Google gives 40 hits for GitHub), which makes it more likely to be memorized by the network. [0]: https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...

It's worth keeping in mind that what a neural network like this (just like GPT3) is doing is generating the most probable continuation based on the training dataset. Not the best continuation (whatever that means), simply the most likely one. If the training dataset has mostly bad code, the most likely continuation is likely to be bad as well. I think this is still valuable, you just have to think before accepting a…

> the most probable continuation based on the training dataset

This is not wrong, but it's easy to misread it as implying little more than a glorified Markov model. If it's like https://www.gwern.net/GPT-3 then it's already significantly cleverer, and so you should expect to sometimes get the kind of less-blatant derivation that companies aim to avoid using a cleanroom process or otherwise forbidding engineers from reading particular sources.

Re: Copilot regurgitating Quake code, including sweary comments

#55

Post by carlmr »

louthy wrote: post #43
bohemian99 wrote: post #32

Earlier quoted context omitted.

My question is would Copilot be useful if you could choose the codebase it would be drawing from? Almost as an internal company tool?

It would certainly alleviate the license concerns. If it was possible to train it to a level (that produces effective output), then sure. As a thought experiment, I thought "what would happen if we trained it on our 15 million lines of product code + my language-ext project". It would almost certainly produce something that looks like 'us'. But: * It would also trip over a million or so lines of generated code * And…

> legacy OO code

Aside from OO vs FP. A concern with that I'd have is that it would encourage and enforce idiosyncracies in large corporate codebases.

If you've ever worked for a large corporation on their legacy code, you know you don't want any of that to be suggested to colleagues.

This would enforce bad behaviors and make it even harder for fresh developers to argue against it.

Re: Copilot regurgitating Quake code, including sweary comments

#56

Post by another-dave »

throwaway287391 wrote: post #27
klohto wrote: post #14

I’m really dumbfounded by the Copilot team decision to not exclude GPL licensed code. Why was this direction chosen? Is the inclusion of GPL really worth the risk and potential Google v. Oracle lawsuit? I’d like to know the reasoning.

Isn't it entirely possible that they did exclude GPL licensed code, but somebody somewhere has violated copyright and copy-pasted that snippet into non-GPL-licensed code that they trained on? They could try to trace every single code snippet they train on to its "true source" and use the license for that, but that's not very well-defined, and is a lot harder, and it's never going to be 100%.

Which raises another question: ideally Copilot wouldn't be trained on "somebody somwhere", but is that happening?

To use the old trope — if the majority of programmers can't implement Fizzbuzz, but they do have a Github profile, are they being included too?

Hopefully there's some quality bar for the training set, i.e. some subset of "good" code (e.g. release candidate tags from fairly established OSS tools/frameworks in different languages) rather than any old code on the internet.

Re: Copilot regurgitating Quake code, including sweary comments

#57

Post by Thomashuet »

thinkingemote wrote: post #3

From the GPLv2 licensed code: https://github.com/id-Software/Quake-III-Arena/blob/master/c... copilot repeats it word for word almost, including comments, and adds an MIT like license up the top

Actually the indentation of the first comment and the lack of preprocessor show it's not copied from this code directly but from Wikipedia (https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...) So It could be that the Quake source code is not part of the training set but the Wikipedia version is.

Re: Copilot regurgitating Quake code, including sweary comments

#58

Post by petercooper »

mimixco wrote: post #16
louthy wrote: post #5

This is utterly damning. I have already instructed my team that Copilot can never be used for our projects. Compromising the product because of unknowable license demands isn't acceptable in the professional world of software engineering. But if we put the licensing to one side for a moment... 1/ Everything I've seen it generate so far is 'imperative hell'. It is practically a 'boilerplate generator'. That might be u…

Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are intro…

While I accept most of the concerns, it's better than your comment suggests. I see some promise for it as a tool for reminding you of a technique or inspiring you to a different approach than you've seen before.

For example, I wrote a comment along the lines of "Find the middle point of two 2D positions stored in x, y vectors" and it came up with two totally different approaches in Ruby - one of which I wouldn't have considered. I did some similar things with SQL, and some people might find huge value in it suggesting regexes, too, because so many devs forget the syntax and a reminder may be all it takes to get out of a jam.

I'm getting old enough now to see where these sorts of prompts will be a game changer, especially when dabbling in languages I'm not very proficient in. For example, I barely know any Python, so I just created a simple list of numbers, wrote a "Sort the numbers into reverse order" comment, and it immediately gave me the right syntax that I'd otherwise have had to Google, taking much longer.

Maybe to alleviate the concerns it could be sandboxed into a search engine or a separate app of its own rather than sitting constantly in my main editor - I would find that a fair compromise which would still provide value but require users to engage in more reflection as to what they're using (at least to a level that they would with using SO answers, say).

Re: Copilot regurgitating Quake code, including sweary comments

#60

Post by nojito »

thinkingemote wrote: post #3

From the GPLv2 licensed code: https://github.com/id-Software/Quake-III-Arena/blob/master/c... copilot repeats it word for word almost, including comments, and adds an MIT like license up the top

It's up to the end user to accept the suggestions.
Post reply on HN