Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

601–610 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#601

Earlier quoted context omitted.

Well... the whole training set is licensed, so you can't really get rid of it. I think that the technology they are using for this is just not ready.

Just retrain the model using properly licensed code? ("just" is doing a ton of heavy lifting, but let's be real, that's not impossibly hard)

There's not many licenses that let you reuse code without including the same headers / licensing blurb. You're in public domain, non-copyleft territory. WTFPL etc.

Re: Copilot regurgitating Quake code, including sweary comments

#602

Earlier quoted context omitted.

Adding to this: I run product security for a large enterprise, and I've already gotten the ball rolling on prohibiting copilot for all the reasons above. It's too big a risk. I'd be shocked if GitHub could remedy the negative impressions minted in the last day or so. Even with other compensating controls around open source management, this flies right under the radar with a c130's worth of adverse consequences.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.

That's actually an attack vector: mirror SO using their open-sourced DB and inject malware into the suggestions, or change the text before it enters the clipboard. People blindly copy/pasting aren't going to notice.

Re: Copilot regurgitating Quake code, including sweary comments

#603

Earlier quoted context omitted.

Apples and oranges: Stack overflow snippets are explicitly granted under a permissive license, as long as you attribute. https://stackoverflow.com/help/licensing It appears that the code that copilot is using is created under a huge variety of licenses, making it risky. On the other hand, a small snippet in a function that is derived from many existing pieces of other code may fall under fair use, even if it is not u…

Stack Overflow and Copilot are similar. Usage of both routinely violates licenses. Stack Overflow content is licensed under CC-BY-SA. Terms [1]: * Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. * ShareAlike — If you remix, transform, or buil…

The difference here is that it's hard to sue a company for sporadic, difficult to track down usages of SO content written by their own engineers.

One can now trivially coerce copilot to regurgitate copyrighted content without attribution. Copilot's basic premise violates the CC-BY-SA terms, and this will continue until no party can demonstrate a viable method of extracting copyrighted code.

There is now a single party backed by a company with a 2 Trillion dollar market cap that can be sued for flagrant copyright violations.

Re: Copilot regurgitating Quake code, including sweary comments

#604

Earlier quoted context omitted.

It just seems bizarre that this wasn’t flagged internally at Microsoft. They have tons of compliance staff.

That's because Microaoft doesn't dare use this for production code (presumably). They are 100% okay with letting their competitors get into legal hot water.

isn't it copilot's liability for "distributing" the copyrighted code?

Re: Copilot regurgitating Quake code, including sweary comments

#605
post #162
post #9

This is pretty clearly just a search engine with more parameters. I thought there was something more going on with copilot, but the fact that it is regurgitating arbitrary code comments tells me that there is zero semantic analysis going on with the actual code being pulled in.

They openly claim it is an AI. What about the state of AI currently in use made you think that there was any intelligence behind it?

When do words lose their meaning? There's the word intelligence in the thing, and yet, no intelligence in the concept itself as known today.

Re: Copilot regurgitating Quake code, including sweary comments

#606
post #465

Earlier quoted context omitted.

I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.

I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious. https://spectrum.ieee.org/tech-talk/artificial-intelligence/... The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.

Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay.

[1] https://docs.github.com/en/github/copilot/about-github-copil...

Re: Copilot regurgitating Quake code, including sweary comments

#607

Earlier quoted context omitted.

Easily offended AI is exactly what the world needs

We have too many easily offended NPC's as is.

Yeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG.

eyeroll

Re: Copilot regurgitating Quake code, including sweary comments

#609
post #440

Earlier quoted context omitted.

The Nazis ran what would more accurately be termed extermination camps. Though what they did certainly bore a strong resemblance to the Boer war concentration camps/manzanar,etc. whose purpose was to "concentrate" people into one place rather than industrially slaughter them.

I don't know if I've ever heard anyone use the term "concentration camp" without qualifiers to refer to anything else than the nazi concentration camps (or something equivalent). Maybe it's just me, but I think it would have been more clear if you said internment camp if your intent was to refer to the broader context and not invoke a comparison to nazis.

I've heard it used in place of internment camp. Though I honestly associate it with the Nazis too.

Re: Copilot regurgitating Quake code, including sweary comments

#610
post #603

Earlier quoted context omitted.

Stack Overflow and Copilot are similar. Usage of both routinely violates licenses. Stack Overflow content is licensed under CC-BY-SA. Terms [1]: * Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. * ShareAlike — If you remix, transform, or buil…

The difference here is that it's hard to sue a company for sporadic, difficult to track down usages of SO content written by their own engineers. One can now trivially coerce copilot to regurgitate copyrighted content without attribution. Copilot's basic premise violates the CC-BY-SA terms, and this will continue until no party can demonstrate a viable method of extracting copyrighted code. There is now a single part…

Surely you would have to sue the people using the tool to produce verbatim copies of code, not the creator of the tool?
Post reply on HN