Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

491–500 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#491

Post by TillE »

canadev wrote: post #464
mediaman wrote: post #428

Earlier quoted context omitted.

Who really copies stack overflow snippets verbatim? It's usually just easier to refer to it for help figuring out the right structure and then adapt it for your own needs. Usually it needs customization for your own application anyway (variables, class instances, etc).

Yeah! I've uh, ... never copied a bit of code into my repo verbatim, right? yeah right. I wish. (Not saying every dev does this)

I've copied plenty of Microsoft sample code verbatim, because the Win32 API sucks and their samples usually get the error handling right.

But, I can't think of a single scenario where I've copied something from Stack Overflow. I'm searching for the idea of how to solve a problem, and typically the relevant code given is either too short to bother copying, or it's long and absolutely not consistent with how I want to write it.

Re: Copilot regurgitating Quake code, including sweary comments

#492

Post by [deleted] »

comex wrote: post #487
gunapologist99 wrote: post #441

Earlier quoted context omitted.

Apples and oranges: Stack overflow snippets are explicitly granted under a permissive license, as long as you attribute. https://stackoverflow.com/help/licensing It appears that the code that copilot is using is created under a huge variety of licenses, making it risky. On the other hand, a small snippet in a function that is derived from many existing pieces of other code may fall under fair use, even if it is not u…

Except that CC-BY-SA is not a permissive license; the SA part is a form of copyleft. It's just that nobody enforces it. From the text [1]: - "[I]f You Share Adapted Material You produce [..] The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-SA Compatible License." - "Adapted Material means material [..] that is derived from or based upon…

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#493

Post by ohazi »

caconym_ wrote: post #405
advisedwang wrote: post #278

Earlier quoted context omitted.

The claim for AI systems like this is that it has actually learned something and is generating code from scratch. Oftentimes the authors will claim regurgitation is simply not possible, and this example shows that's a lie. Many arguments on the benefits, legality and power of AI systems rely on this claim. To turn around now and say it's OK to regurgitate in the right setting is to move the goalposts.

> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowle…

Nat Friedman explicitly stated that it shouldn't regurgitate [0]:

> It shouldn't do that, and we are taking steps to avoid reciting training data in the output

He's being woefully naive. To put it bluntly, we don't know how to build a neural network that isn't capable of spitting out training data. The techniques he pointed to in other threads are academic experiments, and nobody seems to have a credible explanation for why we should believe that they work.

[0] https://news.ycombinator.com/item?id=27677177

Re: Copilot regurgitating Quake code, including sweary comments

#494

Post by caconym_ »

sombremesa wrote: post #488
caconym_ wrote: post #483

Earlier quoted context omitted.

I think that until somebody shows that Copilot is willing to copy distinctive code fragments verbatim, unprompted, with a high occurrence rate, I'm not going to start accusing Github of building an engine to cynically exploit the IP rights of open source copyright holders for profit. I've seen no evidence of that, and in absence of evidence I prefer to remain neutral and open-minded. How would that work, anyway? Rare…

It looks like you're responding to the wrong comment. I don't recall alleging that Github is "building an engine to cynically exploit the IP rights of open source copyright holders for profit".

> I think only Github's lawyers would interpret what GP posted the way you did. Looks like weasel wording to make such an interpretation possible,

So what are you suggesting here, except that Github is attempting a legal sleight-of-hand to hide real infringement?

> while making customers believe that code is more or less synthesized in realtime.

What are you suggesting here except that Github is (essentially) lying to customers, making them believe something that is substantially untrue?

When I say "building an engine to cynically exploit the IP rights of open source copyright holders for profit", I am talking about a scenario in which they are sweeping legitimate IP concerns under the rug with bad faith legal weaselry and misrepresentation of how the product functions, etc., to chase profit. I do not see how that is substantially different from the implications of your comment, especially in the context of this subthread.

Could you enlighten me as to how your intended meaning substantially differs from my interpretation? If you don't mean to accuse Github of malfeasance, we probably don't have much to discuss.

Re: Copilot regurgitating Quake code, including sweary comments

#496

Post by jhgb »

TeMPOraL wrote: post #326
ticviking wrote: post #251

Earlier quoted context omitted.

I'm not really sure that type of tool could really be anything else. How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time?

> I'm not really sure that type of tool could really be anything else. It can't be, because they've chosen to use a deep learning approach. That makes it a dead end right from the start. > How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time? A lot of things that we call "edge cases" are only a problem for humans. They'…

> The way a hypothetical, better Copilot could work, is by having directly encoded grammars and semantics metadata corresponding to popular languages and tools. It could generate code in principled and introspectable way, by having a model of the computation it wants to express and encoding it in a target language.

But that sounds like too much work, let's just throw a lot of data into an NN and see what comes out! /s

> and introspectable

Which most importantly means "debuggable", I assume. From what I get there doesn't seem to be any way to ad-hoc fix an NN's output.

Re: Copilot regurgitating Quake code, including sweary comments

#497

Post by junon »

SahAssar wrote: post #452
pydry wrote: post #440

Earlier quoted context omitted.

The Nazis ran what would more accurately be termed extermination camps. Though what they did certainly bore a strong resemblance to the Boer war concentration camps/manzanar,etc. whose purpose was to "concentrate" people into one place rather than industrially slaughter them.

I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent). Maybe it's just me, but I think it would have been more clear if you said internment camp if your intent was to refer to the broader context and not invoke a comparison to nazis.

Historians themselves call what ICE is doing a concentration camp. So your experience is very much localized.

Re: Copilot regurgitating Quake code, including sweary comments

#498

Post by caconym_ »

ohazi wrote: post #493
caconym_ wrote: post #405

Earlier quoted context omitted.

> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowle…

Nat Friedman explicitly stated that it shouldn't regurgitate [0]: > It shouldn't do that, and we are taking steps to avoid reciting training data in the output He's being woefully naive. To put it bluntly, we don't know how to build a neural network that isn't capable of spitting out training data. The techniques he pointed to in other threads are academic experiments, and nobody seems to have a credible explanation…

"Shouldn't" isn't the same as "doesn't".

I'm not anything close to an ML expert, and I have no opinion on whether what they're aiming for is possible, but this document^[1] (linked in your linked comment) states explicitly that they are aware of the recitation issue and are taking steps to mitigate it. So, in the context of the comment I replied to, I think Github is very far from claiming that recitation is "simply not possible".

^[1] https://docs.github.com/en/github/copilot/research-recitatio...

Re: Copilot regurgitating Quake code, including sweary comments

#499

Post by hdhjebebeb »

SahAssar wrote: post #452
pydry wrote: post #440

Earlier quoted context omitted.

The Nazis ran what would more accurately be termed extermination camps. Though what they did certainly bore a strong resemblance to the Boer war concentration camps/manzanar,etc. whose purpose was to "concentrate" people into one place rather than industrially slaughter them.

I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent). Maybe it's just me, but I think it would have been more clear if you said internment camp if your intent was to refer to the broader context and not invoke a comparison to nazis.

It seems like a distinction without a difference, this article for example uses them interchangably: https://www.commondreams.org/views/2019/06/21/brief-history-...

Re: Copilot regurgitating Quake code, including sweary comments

#500

Post by nitrogen »

greyfox wrote: post #353
nitrogen wrote: post #347

Earlier quoted context omitted.

It would be nice if there were a way to automate the "remembering what that one function is called and what order the parameters are in" portion of my job. IME the best thing for this is looking at the method listing in the docs for the classes I'm using. E.g. for Ruby, it's usually looking at the methods in Enumerable, Enumerator, Array, or Hash. Or I'll drop a binding.pry into the function, run it, and then type ls…

this sounds super interesting, is there a video or upload somewhere that i can watch this being performed in real time?

I very briefly show some of the interactivity of Ruby+Pry here: https://youtu.be/Gy7l_u5G928?t=805 (the overall code segment starts at https://www.youtube.com/watch?v=Gy7l_u5G928&t=626s)

I'd be happy to hear about better demonstrations, and there's also Pry's website (https://pry.github.io/) where they link to some screencasts.

Post reply on HN