Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot regurgitating Quake code, including sweary comments

twitter.com

Post reply on HN
Print view View original

561–570 of 672 posts

Re: Copilot regurgitating Quake code, including sweary comments

#561

Post by kevin_thibedeau »

maclockard wrote: post #453
__MatrixMan__ wrote: post #431

Earlier quoted context omitted.

Is it still a legal concern if I'm just coding because I want to solve a problem and I'm not trying to use it to do business?

If you publish the code anywhere, potentially. You could be (unknowingly) violating the original license if the code was copied verbatim from another source. How much of a concern this is depends heavily on what the original source was.

Distributing binaries to third parties is enough to trigger a license violation. For internal corporate tools, it would be less of an issue as "distribution" hasn't happened.

Re: Copilot regurgitating Quake code, including sweary comments

#562

Post by stingraycharles »

derefr wrote: post #465
Closi wrote: post #385

Earlier quoted context omitted.

Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.

I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.

I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious.

https://spectrum.ieee.org/tech-talk/artificial-intelligence/...

The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.

Re: Copilot regurgitating Quake code, including sweary comments

#563

Post by computerex »

abeppu wrote: post #417

I may be over-reading, but I think this kind of example not only demonstrates the pragmatic legal issues, but also the fundamental weaknesses of a solely text-oriented approach to suggesting code. It doesn't really seem to have a representation of the problem being solved, or the relationship between things it generates and such a goal. This is not surprising in a tool which claims to work at least a little for almos…

I think it’s pretty clear that program synthesis good enough to replace programmers requires AGI.

This solely text based approach is simply “easy” to do, and that’s why we see it. I think it’s cool and results are intriguing but the approach is fundamentally weak and IMO breakthroughs are needed to truly solve the problem of program synthesis.

Re: Copilot regurgitating Quake code, including sweary comments

#564

Post by SSLy »

armatav wrote: post #433
handrous wrote: post #344

Earlier quoted context omitted.

1) Write a project heavily using Copilot (hell, automate it and write thousands of them, why not?) 2) AGPL all that code. 3) Search for large chunks of code very similar to yours, but written after yours, licensed more liberally than AGPL. Ideally in libraries used by major companies. 4) Point the offenders to your repos and offer a "convenient" paid dual-license to make the offenders' code legal for closed-source us…

6) Arms race with someone who trained an obfuscation version that goes through your AGPL code and tweaks it to not be in violation.

I love living in cyberpunk already.

Re: Copilot regurgitating Quake code, including sweary comments

#565

Post by gmueckl »

bencollier49 wrote: post #459
qayxc wrote: post #369

Earlier quoted context omitted.

Since the information is encoded in model weights, I doubt that erasure is even possible. Only post-retrieval filtering would be an option. It only goes to show that intransparent black-box models have no place in the industry. The networks leak information left and right, because it's way too easy to just crawl the web and throw terabytes of unfiltered data at the training process.

> Only post-retrieval filtering would be an option. And illegal, if the original information remains. I assume that there must be a process for altering the training data set and rerunning the entire thing.

The problem is that the information is in an opaque encoding that nobody can reverse engineer today. So it's impossible to prove that a certain subset of data has been removed from the model.

Say, you have a model that repeats certain PII when prompted in a way that I figure out. I show you the prompt, you retrain the model to give a different, non-offensive answer. But now I go and alter the prompt and the same PII reappears. What now?

Re: Copilot regurgitating Quake code, including sweary comments

#566

Post by krick »

minimaxir wrote: post #346
celeritascelery wrote: post #44

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

In my testing of Copilot, the content filters only work on input , not output. Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.

Lol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.

Re: Copilot regurgitating Quake code, including sweary comments

#567

Post by computerex »

bencollier49 wrote: post #459
qayxc wrote: post #369

Earlier quoted context omitted.

Since the information is encoded in model weights, I doubt that erasure is even possible. Only post-retrieval filtering would be an option. It only goes to show that intransparent black-box models have no place in the industry. The networks leak information left and right, because it's way too easy to just crawl the web and throw terabytes of unfiltered data at the training process.

> Only post-retrieval filtering would be an option. And illegal, if the original information remains. I assume that there must be a process for altering the training data set and rerunning the entire thing.

Yes, but the compute costs required for training are probably in the range of hundreds of thousands of usd to potentially millions of usd. Not to mention potentially months of training time.

Re: Copilot regurgitating Quake code, including sweary comments

#569

Post by eximius »

rgbrenner wrote: post #284

So this makes it official... this post[0] and the comments on the announcement[1] concerned about licensing issues were absolutely correct... and this product has the possibility of getting you sued if you use it. Unfortunately for GitHub, there's no turning back the clocks. Even if they fix this, everyone that uses it has been put on notice that it copies code verbatim and enables copyright infringement. Worse, ther…

Seems like the liability should also be on Copilot itself, as a derivative work.

Re: Copilot regurgitating Quake code, including sweary comments

#570

Post by krick »

anyonecancode wrote: post #101

I think copilot is solving the wrong problem. A future of programming where we're higher up the abstraction tree is absolutely something I want to see. I am taking advantage of that right now -- I'm a decently good programmer, in the sense that I can write useful, robust, reliable software, but I'm pretty high up the stack, working in languages like Java or even higher up the stack that free me from worrying about th…

Yeah, but generating tons of stupid verbose code that nobody will be able to read and understand is more fun. Also, your superiors will be sure you are a valuable worker if you write more code.
Post reply on HN