Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

251–260 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#251

Post by ticviking »

foobiekr wrote: post #226
tyingq wrote: post #30

They have 4 hand picked examples on their homepage: https://copilot.github.com/ One has the issue with form encoding: https://news.ycombinator.com/item?id=27697884 The python example is using floats for currency, in an expense tracking context. The golang one uses a word ("value") for a field name that's been a reserved word since SQL-1999. It will work in popular open source SQL databases, but I believe it would bom…

Copilot appears to be “give more efficiency leverage to the worst kind of coder.”

I'm not really sure that type of tool could really be anything else.

How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time?

Re: Copilot regurgitating Quake code, including sweary comments

#252

Post by user-the-name »

aj3 wrote: post #210
FemmeAndroid wrote: post #161

Earlier quoted context omitted.

But Copilot won't accurately tell you if it's directly copying code, and if so what the license is. If it provides MIT licensed code that I then need to include, how do I know that? Do I need to search for each set of lines of code it provides on GitHub? When a person gets code from another source on the internet, they generally know where the code has come from.

In a real world scenario you wouldn't be mindlessly pressing Tab right after linebreak and accepting the first suggestion that comes your way. While entertaining, nobody gets paid to do that. What you get paid is to write your own code. When you write your own code, generally you think first and then type. Well, with Copilot you think first and then start typing a few symbols before seeing automatic suggestions. If t…

But it is not the same as if you typed it yourself.

If you happen to type code that is similar to copyright code, that is generally considered legally OK.

If you copypaste copyrighted code, that is not legally OK.

If you accept that same code from an autocomplete tool, that can easily be seen as equivalent to the latter case rather than the former.

Re: Copilot regurgitating Quake code, including sweary comments

#253

Post by aj3 »

cjaybo wrote: post #228
aj3 wrote: post #124

Earlier quoted context omitted.

Then write the code yourself. It's not like you're forced to use this demo.

Not exactly a confidence-inspiring reply from someone who just identified themselves as representing the project here!

I don't work for Github (nor MS) and do not represent Copilot.

Re: Copilot regurgitating Quake code, including sweary comments

#254

Post by School-Cotton »

tyingq wrote: post #207
School-Cotton wrote: post #184

Earlier quoted context omitted.

> Can you give an example? The answer is the same as _any_ time you should use floats: where you don't care about answers being exact, either (1) because calculation speed is more important than exactness, or (2) because your inputs or computations involve uncertainty anyway, so it doesn't matter. This is more likely to be the case in, say, physics than it is in finance, but it's not impossible in the latter. For exa…

That's fair, though the example code I mentioned is the start of an expense tracker.

Fair enough -- in that case, you should definitely use either a decimal type or an integer.

Re: Copilot regurgitating Quake code, including sweary comments

#256

Post by vmception »

slumdev wrote: post #169
vmception wrote: post #76

Earlier quoted context omitted.

Hm odd takes here. It's really weird for software engineers to judge something by its current state and not by its potential state. To me, it's clearly solvable by Copilot filtering the input code by that repository's license. It should only be certain open source licenses, maybe even user-choosable, or code-creators can optionally sublicense their code to Copilot in a very permissable way. Secondly, a way for the cr…

> It's really weird for software engineers to judge something by its current state and not by its potential state. No, we're not afraid of Copilot replacing us. The thought is ridiculous, anyway. If it actually worked, we would be enabled to work in higher abstractions. We'd end up in even higher demand because the output of a single engineer would be so great that even small businesses would be able to afford us. Ye…

Sounds like projecting because thats not what I was referring to

I’m saying copilot can be better with very simple tweaks

Re: Copilot regurgitating Quake code, including sweary comments

#257

Post by gmadsen »

ticviking wrote: post #251
foobiekr wrote: post #226

Earlier quoted context omitted.

Copilot appears to be “give more efficiency leverage to the worst kind of coder.”

I'm not really sure that type of tool could really be anything else. How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time?

a large data set covering exactly what you just mentioned?

Re: Copilot regurgitating Quake code, including sweary comments

#258

Post by joeyh »

buu700 wrote: post #181
gpm wrote: post #160

Earlier quoted context omitted.

Maybe? Might depend on the license, I doubt the courts would be amused. Almost certainly a link would not suffice, basically every license requires that the attribution be directly included with the modified material. Links can rot, can be inaccessible if you don't have internet access, can change out from underneath you, etc. (I am not a lawyer, btw)

Makes sense. Maybe something like git-lfs/git-annex would be sufficient to address the linking issue, but it seems like the bigger concern is whether a court would accept this as valid attribution. In a sense it reminds me of the LavaBit stunt with the printed key.

I think a judge could be persuaded that a list of every known human does not constitute a valid attribution of the actual author, even though their name is on the list. The purpose of an attribution is to acknowledge the creator of the work, and such a list fails at that.

Re: Copilot regurgitating Quake code, including sweary comments

#259

Post by onion2k »

mimixco wrote: post #193
onion2k wrote: post #45

Earlier quoted context omitted.

Stuff like Copilot makes programming worse and more difficult, not better and easier. Copilot makes programming worse and more difficult if you're aiming for a specific set of coding values and style that Copilot doesn't generate (yet?). If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. The author of a…

I don't think they work out past trivial applications. Any non trivial app requires an understanding of a much larger part of the codebase than a tool like Copilot is looking at at any one time. Copilot does not understand the code in toto and is therefore really useless for debugging (70% of all coding) and probably useless for anything other than very simple parts of an app.

Any non trivial app requires an understanding of a much larger part of the codebase than a tool like Copilot is looking at at any one time.

I don't think that's important. Copilot, at least as it's been demo'd so far judging by the examples, is to help you write small, standalone functions. It shouldn't need to know about the rest of the application. Just as the functions that you write yourself shouldn't need to know about the rest of the application either.

If your functions need a broad understanding of the codebase as a whole how the heck do you write tests that don't fail the instant anything changes?

Re: Copilot regurgitating Quake code, including sweary comments

#260

Post by visarga »

LeanderK wrote: post #71
avian wrote: post #31

Earlier quoted context omitted.

Famous code examples are also much more likely to be noticed. For all I know, the thing might be spewing random GPL'd code from the long tail of GitHub all the time and nobody notices because it was written by some random guy and not John Carmack.

Well, it's sure speculation on my part what the root cause is, but i think OpenAI is already trying to ensure the network generalises. It's just common behaviour for neural network to memorise frequent samples, so I think my guess is quite realistic. I don't think OpenAI would not notice large-scale memorisation in their model. But as long as they don't publish more details it's just guesswork. Just keep in mind that…

Hash 10-grams and make a bloom filter. It will not generate more than 10 GPLed tokens from a source.
Post reply on HN