Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

531–540 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#531

Post by bartread »

mediaman wrote: post #428
fragmede wrote: post #420

Earlier quoted context omitted.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.

Who really copies stack overflow snippets verbatim? It's usually just easier to refer to it for help figuring out the right structure and then adapt it for your own needs. Usually it needs customization for your own application anyway (variables, class instances, etc).

Ha! Well, I think a lot of people copy code from StackOverflow verbatim once at least - including me.

Of course it turned out the code I'd blindly inserted into my project contained a number of bugs. In one or two cases, quite serious ones. This, even though it was the accepted answer.

It was probably more effort to fix up the code I'd copy pasta'd than write it from scratch. Since then I've never copied and pasted from StackOverflow verbatim.

Re: Copilot regurgitating Quake code, including sweary comments

#532

Post by bambax »

dragonwriter wrote: post #450
SahAssar wrote: post #435

Earlier quoted context omitted.

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism. US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

Why is this downvoted... It's simply the truth.

Re: Copilot regurgitating Quake code, including sweary comments

#533

Post by TheDong »

dr_kiszonka wrote: post #345
croes wrote: post #79

Earlier quoted context omitted.

Good luck checking every code line for license violations

There will be a VSCode extension for that.

It's impossible to automate checking for code license violations.

If you and I write the exact same 10 lines of code, we both have independent and valid copyrights to it. Unlike patents, independent derivation of the same code _is_ a defense for copyright.

If I write 10 lines of code, publish it as GPL (but don't sign a CLA / am not assigning it to an employer), and then re-use it in an MIT codebase, I can do that because I retained copyright, and as the copyright holder I can offer the code under multiple incompatible licenses.

There's no way for a machine to detect independent derivation vs copying, no way for the machine to know who the original copyright holder was in all cases, and whether I have permission from them to use it under another license (i.e. if I email the copyright holder and they say 'yeah, sure, use it under non-gpl', it suddenly becomes legal again)...

It's not a problem computers can solve 100% correctly.

Re: Copilot regurgitating Quake code, including sweary comments

#534

Post by influx »

SEMW wrote: post #177
influx wrote: post #20

I get why marketing calls machine learning “AI”. I don’t get why engineers would think this is. Dumb.

> I don’t get why engineers would think this is. This claim that "AI" only means artificial general / human-equivalent intelligence completely ignores the long history of how that term has been used, by computer science researchers, for the last 70-odd years, to include everything from Shannon's maze-solving algorithms, to Prolog-y systems, to simple reinforcement learning, and so on. https://web.archive.org/web/2007…

It hasn't been AI the entire time. It's borderline fraud, tbh.

Re: Copilot regurgitating Quake code, including sweary comments

#535

Post by [deleted] »

axiosgunnar wrote: post #138

I hate to be the one that says this but I think it‘s true: "So you are an SWE and you take a break from work to go to Hackernews to complain that Github's Copilot, which is an AI-based solution meant to help SWEs, is utter shit and completely unusuable. And then you go back to writing AI-based solutions for some other profession. Which is totally not shit or anything.“ Can anybody put this more elegantly?

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#536

Post by blindmute »

throwaway2037 wrote: post #387
celeritascelery wrote: post #44

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

Quoted post unavailable.

> After explaining the historical context, 99% of people reply: "Oh, I understand. Thank you to explain."

A similar percentage then think to themselves, privately, "well that's pretty stupid."

Re: Copilot regurgitating Quake code, including sweary comments

#537

Post by pmkiwi »

pydry wrote: post #440
SahAssar wrote: post #435

Earlier quoted context omitted.

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

The Nazis ran what would more accurately be termed extermination camps. Though what they did certainly bore a strong resemblance to the Boer war concentration camps/manzanar,etc. whose purpose was to "concentrate" people into one place rather than industrially slaughter them.

To be correct, both existed.

A camp like Ravensbrück was a concentration camp (for women) while Auschwitz-Birkenau was both a concentration and extermination camp.

https://upload.wikimedia.org/wikipedia/commons/b/be/WW2_Holo...

Re: Copilot regurgitating Quake code, including sweary comments

#538

Post by kanzenryu2 »

dragonwriter wrote: post #450
SahAssar wrote: post #435

Earlier quoted context omitted.

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism. US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

There were only a handful of mass extermination camps. There were tens of thousands of concentration camps. https://encyclopedia.ushmm.org/content/en/article/nazi-camps....

Re: Copilot regurgitating Quake code, including sweary comments

#539

Post by joe_the_user »

macNchz wrote: post #218
gruez wrote: post #26

Earlier quoted context omitted.

Offensive code is the least of my worries. What about vulnerable/exploitable code?

This was my first thought when reading about Copilot...it feels almost certain that someone will try poisoning the training data. Hard to say how straightforward it'd be to get it to produce consistently vulnerable suggestions that make it into production code, but I imagine an attacker with some resources could fork a ton of popular projects and introduce subtle bugs. The sentiment analysis example on the Copilot la…

Targeted attacks to elicit output only at a give context are generally possible with AIs. And here, writing an implementation of a difficult and vulnerable process seems easy. Bad implementations of various hard things become common 'cause people cut and paste the code without looking closely since they don't understand it anyway.

//Implement eliptic cryptography below

//Sanitize input for SQL call below

Etc.

Re: Copilot regurgitating Quake code, including sweary comments

#540

Post by caconym_ »

ohazi wrote: post #530
caconym_ wrote: post #517

Earlier quoted context omitted.

You're using some pretty strong language here, but do you have any more substantive criticisms of the analysis they present at https://docs.github.com/en/github/copilot/research-recitatio... ? They seem to think the incidence of meaningful (i.e. substantively infringing) recitation is very low, and that their solution in those cases will be attribution rather than elimination. Again, I'm not an ML expert, but that so…

They had some people use the thing for a while, and concluded "Hey look, it doesn't seem to quote verbatim very often. Yay!" There is nothing in there that describes any sort of mitigation. The three sentences about an attribution search at the very end are aspirational at best, and are presented as "obvious" even though it's not at all clear that such a fuzzy search can be implemented reliably. I use the halting pro…

> The analysis you're citing is just that -- a statistical analysis. They had some people use the thing for a while, and concluded "Hey look, it doesn't seem to quote verbatim very often. Yay!" There is nothing in there that describes any sort of mitigation.

> The three sentences about an attribution search at the very end are aspirational at best, and are presented as "obvious" even though it's not at all clear that such a fuzzy search can be implemented reliably.

I agree with all of this, though I do think that the attribution strategy they describe sounds a lot easier than solving the halting problem or entirely eliminating recitation in their model. Obviously, the proof will be in the pudding.

Maybe you and others are reacting to them framing this as "research", as if they're trying to prove some fundamental property of their model rather than simply harden it against legally questionable behavior in a more practical sense. I think a statistical analysis is fine for the latter, assuming the sample is large enough.

Post reply on HN