Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

481–490 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#481

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

Quoted post unavailable.

I don't work in USA and I don't intend to. Your history of slavery is none of my concern, especially when I'm just trying to do my work.

The word 'master' is useful for me, and I don't believe for a nanosecond that anyone, American or not, is ACTUALLY offended by it. I believe that some people (mostly affluent white Americans) are searching for things that they think they SHOULD be offended by.

Re: Copilot regurgitating Quake code, including sweary comments

#482
post #418
post #362

Earlier quoted context omitted.

> Python not really as the language doesn't provide any way to keep invariants Again, how is that relevant? If there's no way to enforce an invariant in custom data types , then there's also no way to enforce invariants in code using built-in data types .

It is surely relevant. Rust provides the mechanisms to enforce them, while in Python, like all dynamic languages, everything is up for grabs.

What I meant [1] was: In Python, invariants are enforced by conventions, not by the compiler. If that's not suitable for a given use case, then Python is entirely unsuited for that use case, regardless whether it provides built-in decimal types or user-defined decimal types. That's why I said that your objection regarding invariant enforcement is irrelevant to this discussion.

[1] (but was to lazy to write out)

Re: Copilot regurgitating Quake code, including sweary comments

#483

Earlier quoted context omitted.

This actually seems like an explicit acknowledgement that regurgitation is possible, and not remotely a claim that it is "simply not possible". It stands to reason that cases where people are intentionally trying to produce regurgitation will strongly overlap with the minority of cases where it actually happens. So I think we are probably suffering from some selection bias in discussions on HN and similar forums--tha…

I think only Github's lawyers would interpret what GP posted the way you did. Looks like weasel wording to make such an interpretation possible, while making customers believe that code is more or less synthesized in realtime. "Snippets" makes one think one or two lines of code, not entire functions and classes.

I think that until somebody shows that Copilot is willing to copy distinctive code fragments verbatim, unprompted, with a high occurrence rate, I'm not going to start accusing Github of building an engine to cynically exploit the IP rights of open source copyright holders for profit. I've seen no evidence of that, and in absence of evidence I prefer to remain neutral and open-minded.

How would that work, anyway? Rare, distinctive code forms seem much more difficult for an ML thing to suggest with a high-ish confidence level, since there won't be much training data. The Quake thing makes sense because it's one of the most famous sections of code in the world, and probably exists in thousands of places in the public Github corpus.

I'm emphasizing distinctive because a lot of boilerplate takes up a lot of room, but still doesn't make a reasonable argument for copyright infringement when yours looks like somebody else's.

Re: Copilot regurgitating Quake code, including sweary comments

#484

I hate to be the one that says this but I think it‘s true: "So you are an SWE and you take a break from work to go to Hackernews to complain that Github's Copilot, which is an AI-based solution meant to help SWEs, is utter shit and completely unusuable. And then you go back to writing AI-based solutions for some other profession. Which is totally not shit or anything.“ Can anybody put this more elegantly?

>And then you go back to writing AI-based solutions for some other profession.

I don't know what you're talking about, I'm a webshit developer.

Re: Copilot regurgitating Quake code, including sweary comments

#485

Earlier quoted context omitted.

It's not so much a regional as a political thing. Want it to sound worse? Use concentration camp. Want it to sound better? Use internment camp (or in some cases, re-education facility).

Or “Reserve”.

Relevant to that, the US WWII internment camps were...placed on land taken from (with disputedly-adequate compensation for the use) reservation land.

Re: Copilot regurgitating Quake code, including sweary comments

#486

Earlier quoted context omitted.

Is it still a legal concern if I'm just coding because I want to solve a problem and I'm not trying to use it to do business?

If you publish the code anywhere, potentially. You could be (unknowingly) violating the original license if the code was copied verbatim from another source. How much of a concern this is depends heavily on what the original source was.

And the problem with copilot is that you have no way of knowing. If it changes even a little bit of the code, it's basically ungoogleable but still potentially in violation.

Re: Copilot regurgitating Quake code, including sweary comments

#487

Earlier quoted context omitted.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.

Apples and oranges: Stack overflow snippets are explicitly granted under a permissive license, as long as you attribute. https://stackoverflow.com/help/licensing It appears that the code that copilot is using is created under a huge variety of licenses, making it risky. On the other hand, a small snippet in a function that is derived from many existing pieces of other code may fall under fair use, even if it is not u…

Except that CC-BY-SA is not a permissive license; the SA part is a form of copyleft. It's just that nobody enforces it. From the text [1]:

- "[I]f You Share Adapted Material You produce [..] The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-SA Compatible License."

- "Adapted Material means material [..] that is derived from or based upon the Licensed Material" (emphasis added)

- "Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.'

- "You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, Adapted Material that restrict exercise of the rights granted under the Adapter's License You apply."

A program that includes a code snippet is unquestionably a derived work in most cases. That means that if you include a Stack Overflow code snippet in your program, and fair use does not apply, then you have to license the entire program under the CC-BY-SA. Alternately, you can license it under the GPLv3, because the license has a specific exemption allowing you to relicense under the GPLv3.

For open source software under permissive licenses, it may actually be okay to consider the entire program as licensed under the CC-BY-SA, since permissive licenses are typically interpreted as allowing derived works to be licensed under different licenses; that's how GPL compatibility works. But you'd have to be careful you don't distribute the software in a way that applies any Effective Technological Measures, aka DRM. Such as via app stores, which often include DRM with no way for the app author to turn it off. (It may actually be better to relicense to the GPL, which 'only' prohibits adding additional terms and conditions, not the mere use of DRM. But people have claimed that the GPL also forbids app store distribution because the app store's terms and conditions count as additional restrictions.)

For proprietary software where you do typically want to impose "different terms or conditions", this is a dead end.

Note that copying extremely short snippets, or snippets which are essentially the only way to accomplish a task, may be considered fair use. But be careful; in Oracle v. Google, Google's accidental copying of 9 lines of utterly trivial code [2] was found to be neither fair use nor "de minimis", and thus infringing.

Going back to Stack Overflow, these kinds of surprising results are why Creative Commons itself does not recommend using its licenses for code. But Stack Overflow does so anyway. Good thing nobody ever enforces the license!

See also: https://opensource.stackexchange.com/questions/6777/can-i-us...

[1] https://creativecommons.org/licenses/by-sa/4.0/legalcode

[2] https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/

Re: Copilot regurgitating Quake code, including sweary comments

#488

Earlier quoted context omitted.

I think only Github's lawyers would interpret what GP posted the way you did. Looks like weasel wording to make such an interpretation possible, while making customers believe that code is more or less synthesized in realtime. "Snippets" makes one think one or two lines of code, not entire functions and classes.

I think that until somebody shows that Copilot is willing to copy distinctive code fragments verbatim, unprompted, with a high occurrence rate, I'm not going to start accusing Github of building an engine to cynically exploit the IP rights of open source copyright holders for profit. I've seen no evidence of that, and in absence of evidence I prefer to remain neutral and open-minded. How would that work, anyway? Rare…

It looks like you're responding to the wrong comment. I don't recall alleging that Github is "building an engine to cynically exploit the IP rights of open source copyright holders for profit".

Re: Copilot regurgitating Quake code, including sweary comments

#489
post #467

Earlier quoted context omitted.

I mean, the cases where it tries to assign copyright to another person in a different year highlights that context other than the other text in the file is semantically extremely important, and not considered by this approach. Merely generating text which looks appropriate to the model given surrounding text is ... misguided? If you think about it, program synthesis is one of the few problems in which the system can…

> generating text which looks appropriate to the model given surrounding text is ... misguided? Agreed - it represents a failure to adequately model/understand the task, but I don't think it is a "fundamental weakness" of text-based 'Chinese room' approaches. > You gotta know the referents, not just the symbols. No one wants a half-broken Chinese Room. "Knowing the referents" is not at all clearly defined. It's total…

> "Knowing the referents" is not at all clearly defined. It's totally possible that, under the constraint of optimizing for next-word prediction, the model could develop an understanding of what the referents are.

Well, in this case, it would have been good to understand that "V. Petkov" is a person unrelated to the project being written, and that "2015" is a year and not the one we're currently in. Sometimes the referent will be a method defined in an external library, which perhaps has a signature, and constraints about inputs, or properties which apply to return values.

> You can't underestimate the level of complex behavior emerging from a big enough system under optimization. After all, all the crazy stuff we do - coding, art, etc. is produced by a system under evolutionary optimization pressure to make more of itself.

I think this can verge into a kind of magical thinking. Yes, humans also look like neural nets, and we might even be optimizing for something. But we learn to program (and we do our best job programming) by having a goal for program behavior, and we use interactive access to try to run something, get an error, set a break point, try again, etc. I challenge anyone to try to learn to "code" by never being given any specific tasks, never interacting with docs about the language, an interpreter, a compiler, etc, but merely to try to fill in the blank in paper code snippets. You might learn to fill in some blanks. I highly doubt you would learn to code.

This is totally a case where the textual representation of programs is easier to get and train against, and that tail is being allowed to wag the dog to frame both the problem and the product.

None of this is to say that high-bandwidth DNN approaches don't have a place here -- but I think we should be looking at language-specific models where the DNN receives information about context (including some partial description of behavior) and outputs of the DNN are something like the weights in a PCFG that is used in the program search.

Re: Copilot regurgitating Quake code, including sweary comments

#490

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

Is it still a legal concern if I'm just coding because I want to solve a problem and I'm not trying to use it to do business?

Yes: not all code on GitHub is licensed in a way that lets you use it at all. People focus on GPL as if that were the tough case; but, in addition to code (like mine) under AGPL (which you need to not use in a product that exposes similar functionality to end users) there is code that is merely published under "shared source" licenses (so you can look, but not touch) and even literally code that is stolen and leaked from the internals of companies--including Microsoft!... this code often gets taken down later, but it isn't always noticed and either way: it is now part of Copilot :/--that, if you use this mechanism, could end up in your codebase.
Post reply on HN