Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

451–460 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#451

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#452
post #440

Earlier quoted context omitted.

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

The Nazis ran what would more accurately be termed extermination camps. Though what they did certainly bore a strong resemblance to the Boer war concentration camps/manzanar,etc. whose purpose was to "concentrate" people into one place rather than industrially slaughter them.

I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent).

Maybe it's just me, but I think it would have been more clear if you said internment camp if your intent was to refer to the broader context and not invoke a comparison to nazis.

Re: Copilot regurgitating Quake code, including sweary comments

#453

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

Is it still a legal concern if I'm just coding because I want to solve a problem and I'm not trying to use it to do business?

If you publish the code anywhere, potentially. You could be (unknowingly) violating the original license if the code was copied verbatim from another source.

How much of a concern this is depends heavily on what the original source was.

Re: Copilot regurgitating Quake code, including sweary comments

#455
post #442

Earlier quoted context omitted.

> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowle…

They did! In the faq which I can't find anymore they said: >GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.

This actually seems like an explicit acknowledgement that regurgitation is possible, and not remotely a claim that it is "simply not possible".

It stands to reason that cases where people are intentionally trying to produce regurgitation will strongly overlap with the minority of cases where it actually happens. So I think we are probably suffering from some selection bias in discussions on HN and similar forums--that might be unavoidable, and it certainly stimulates some interesting discussion, but we should try to avoid misrepresenting the product as a whole and/or what its creators have said about it.

Re: Copilot regurgitating Quake code, including sweary comments

#456
post #325

Copilot was only trained on public code right? Because that would be an absolute PR disaster if some proprietary code was leaked that way.

I'm curious if you can find certain keywords that may leak private code...

Waiting for the access to test this out

Re: Copilot regurgitating Quake code, including sweary comments

#457

I've always assumed that we would eventually have a low-code, or no-code junior dev replacement, and was wondering if this was it. GH and MS actually have [Ed. had?] some cred for this kind of thing. Nope. Game over. Play again?

Most low-code and no-code platforms go for junior dev empowerment, and senior dev replacement. This one also seems to be aimed at empowering juniors, but looks like it missed the senior replacement by miles.

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#458

Earlier quoted context omitted.

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism. US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

Sure, but I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent).

If someone says that something is "_literally_ a concentration camp" I think that most people will think of ovens and genocide.

Perhaps it's a regional thing, but that is how I interpreted it.

Re: Copilot regurgitating Quake code, including sweary comments

#459
post #369

Earlier quoted context omitted.

That's bonkers. And the beauty of it is that now someone could realistically do a GDPR Erasure request on the Neural Net. I do hope that they're able to reverse data out.

Since the information is encoded in model weights, I doubt that erasure is even possible. Only post-retrieval filtering would be an option. It only goes to show that intransparent black-box models have no place in the industry. The networks leak information left and right, because it's way too easy to just crawl the web and throw terabytes of unfiltered data at the training process.

> Only post-retrieval filtering would be an option.

And illegal, if the original information remains.

I assume that there must be a process for altering the training data set and rerunning the entire thing.

Re: Copilot regurgitating Quake code, including sweary comments

#460

From the GPLv2 licensed code: https://github.com/id-Software/Quake-III-Arena/blob/master/c... copilot repeats it word for word almost, including comments, and adds an MIT like license up the top

I guess this confirms John Carmack to be an AI

I get why some people were saying it made them a better programmer. Of course it did, it's copy-pasting Carmack code.
Post reply on HN