Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

441–450 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#441

Post by gunapologist99 »

fragmede wrote: post #420
eganist wrote: post #384

Earlier quoted context omitted.

Adding to this: I run product security for a large enterprise, and I've already gotten the ball rolling on prohibiting copilot for all the reasons above. It's too big a risk. I'd be shocked if GitHub could remedy the negative impressions minted in the last day or so. Even with other compensating controls around open source management, this flies right under the radar with a c130's worth of adverse consequences.

Do you also block stack overflow and give guidance to never copy code from that website or elsewhere on the Internet? I'm legitimately curious - my org internally officially denounces the copying of stack overflow snippets. Thankfully for my role it's moot as I mostly work with an internal non-public language, for better or worse, and I have no idea how well that's followed elsewhere in the wider company.

Apples and oranges: Stack overflow snippets are explicitly granted under a permissive license, as long as you attribute.

https://stackoverflow.com/help/licensing

It appears that the code that copilot is using is created under a huge variety of licenses, making it risky.

On the other hand, a small snippet in a function that is derived from many existing pieces of other code may fall under fair use, even if it is not under an open source license of some sort.

Re: Copilot regurgitating Quake code, including sweary comments

#442

Post by rckstr »

caconym_ wrote: post #405
advisedwang wrote: post #278

Earlier quoted context omitted.

The claim for AI systems like this is that it has actually learned something and is generating code from scratch. Oftentimes the authors will claim regurgitation is simply not possible, and this example shows that's a lie. Many arguments on the benefits, legality and power of AI systems rely on this claim. To turn around now and say it's OK to regurgitate in the right setting is to move the goalposts.

> Oftentimes the authors will claim regurgitation is simply not possible Do the Copilot authors claim this? I get that you're suggesting that Copilot may benefit from absolute claims made by the authors of other, similar systems (or their proponents), but I also don't think it's reasonable to exclude nuance and the specifics of Copilot from ongoing discussions on that basis. The Copilot authors have publicly acknowle…

They did! In the faq which I can't find anymore they said:

>GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.

Re: Copilot regurgitating Quake code, including sweary comments

#443

Post by amelius »

rubatuga wrote: post #339
amelius wrote: post #270

Earlier quoted context omitted.

What are the statistics of Copilot based on a validation set? How often does it get code right? I want to see hard statistics, not 4 hand-picked examples.

Yeah. And like how would you even devise a metric? Like compile it down to assembly and see if it’s similar logic?

Well, this is the question which the producers of the tool should answer.

You can't just release a ML tool onto the public if you haven't validated it first.

Re: Copilot regurgitating Quake code, including sweary comments

#444

Post by slackfan »

throwaway2037 wrote: post #387
celeritascelery wrote: post #44

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

Quoted post unavailable.

And in my historical context power fists that your ideology used were used by a regime that murdered millions. In the past 100 years.

Re: Copilot regurgitating Quake code, including sweary comments

#445

Post by spoonjim »

celeritascelery wrote: post #44

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

Blocks offensive words, but doesn't block carefully crafted malware.

Re: Copilot regurgitating Quake code, including sweary comments

#448

Post by Haga »

OskarS wrote: post #290
arksingrad wrote: post #211

Earlier quoted context omitted.

I guess this confirms John Carmack to be an AI

Apparently Carmack was not the original author, the origin I believe is SGI somewhere in the deep dark 90s.

Was a optimization for a fluid simulation originally..

Re: Copilot regurgitating Quake code, including sweary comments

#449

Post by [deleted] »

fragmede wrote: post #422
mmastrac wrote: post #403

Earlier quoted context omitted.

Well... the whole training set is licensed, so you can't really get rid of it. I think that the technology they are using for this is just not ready.

Just retrain the model using properly licensed code? ("just" is doing a ton of heavy lifting, but let's be real, that's not impossibly hard)

[deleted]

Re: Copilot regurgitating Quake code, including sweary comments

#450

Post by dragonwriter »

SahAssar wrote: post #435
pydry wrote: post #408

Earlier quoted context omitted.

Changing master to main was something Github did when they were taking heat for their contract with ICE. It was a nice bit of misdirection that cost them nothing, achieved nothing and garnered praise in some quarters. ICE, of course, runs an actual concentration camp which has a slightly more troublesome history than the word master. Language policing is to racism what recycling is to global warming - an attempt to s…

I get what you mean, but in a discussion about semantics it might be unhelpful to dilute the term "concentration camp", especially if prefixed with "actual" in italics. That is unless you actually mean that ICE camps serve the same purpose and are equivalent to nazi concentration camps.

Nazi “concentration camps” were not actual concentration camps (a thing which long predates the Nazi camps), they were extermination camps for which “concentration camp” was a minimizing euphemism.

US WWII “internment” and “relocation” centers were actual concentration camps (“relocation center” was itself a euphemism, but “internment” referred to a formal legal distinction impacting treaty obligations.)

Post reply on HN