Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All public GitHub code was used in training Copilot

twitter.com

Post reply on HN
Print view View original

71–80 of 734 posts

Re: All public GitHub code was used in training Copilot

#71

Post by [deleted] »

wongarsu wrote: post #44
kzrdude wrote: post #18

Earlier quoted context omitted.

The training question seems much more difficult. The main problem that has been the topic is a simpler one - about the produced work. If you exactly reproduce someone's existing code (doesn't matter if you copy by flipping bits one by one or which technology you use), isn't it a copyright violation? I'm kind of imagining a Rube Goldberg machine that spells out the quake invsqrt function in the sand, now...

Yes, if you play a video from Netflix while recording your screen, transcode that video to MPEG2 and use a red laser to write a complex encoding of that MPEG2 bitstream onto a plastic disk, then send that by mail to your friend, a court won't care about the complexity of that Rube Goldberg machine. They will just say it's a clear copyright violation since you distributed a Netflix movie by DVD. With programming, ther…

[deleted]

Re: All public GitHub code was used in training Copilot

#72

Post by leereeves »

undecisive wrote: post #45

I see a lot of people trying to compare its "machine learning" to human learning. Let's use this thought experiment: Imagine that Github's Copilot was just a massive array of all the lines of code from every github project, with some (magical automated whatever) tagging and indexing on each function, and a search engine on top of that. Now imagine that copilot simply finds the closest search result, and then when you…

I don't think it matters where Copilot is on that line. A skilled human programmer at the far end of that line, fully capable of producing novel programs that they haven't seen before, would still be violating copyright if they reproduced a program they have seen before.

Re: All public GitHub code was used in training Copilot

#73

Post by ghoward »

saint_abroad wrote: post #23
0x0 wrote: post #7

To the people arguing it's "fair use" to use this for training an ML network. Where do you draw the line? What if you train an "ML network" with one or two inputs... so that they almost always "generate" exact copies of the inputs? Five inputs..? Ten? A thousand? A million?

> Where do you draw the line? My simplistic view is that the following is legally equivalent: input -> ai network -> output input -> huffman coding -> output So, whilst: * compressing and decompressing a copyright work is permissible; * output and weights are deterministic transformations of the inputs; thus: * not eligible for copyright (lacking creativity); and * are derivative works of the inputs;

But at the same time, a compiler does a deterministic transformation of its inputs, and we still count its output as under copyright and license.

copyrighted input -> compiler -> copyrighted output

Re: All public GitHub code was used in training Copilot

#74

Post by Tenoke »

mikehearn wrote: post #66

ML novice question: is this atypical when training models? Wasn't GPT-3 trained on a lot of copyrighted data? My gut instinct, which is based on very low-information, is that it would be pretty hard to train models if you could only use open-licensed material.

Yes, it would stiffle NLP research immensely and we likely wouldn't see anything better than gpt3 for years if such restrictions are put in place.

Re: All public GitHub code was used in training Copilot

#75

Post by jonfw »

mikehearn wrote: post #66

ML novice question: is this atypical when training models? Wasn't GPT-3 trained on a lot of copyrighted data? My gut instinct, which is based on very low-information, is that it would be pretty hard to train models if you could only use open-licensed material.

It would be pretty concerning if people used GPT-3 while they were writing a novel, and it assisted them in plagiarizing a Steven King novel.

We already have examples of copilot blatantly plagiarizing code

Re: All public GitHub code was used in training Copilot

#76

Post by speedgoose »

ghoward wrote: post #70

This is why I relicensed my code [1] yesterday to a license I wrote [2], which is designed to poison the well for machine learning. [1]: https://gavinhoward.com/2021/07/poisoning-github-copilot-and... [2]: https://yzena.com/yzena-network-license/

Since you allow new versions by default, can't someone just release a new version of your license allowing everything they want?

Re: All public GitHub code was used in training Copilot

#77

Post by lumost »

Tenoke wrote: post #74
mikehearn wrote: post #66

ML novice question: is this atypical when training models? Wasn't GPT-3 trained on a lot of copyrighted data? My gut instinct, which is based on very low-information, is that it would be pretty hard to train models if you could only use open-licensed material.

Yes, it would stiffle NLP research immensely and we likely wouldn't see anything better than gpt3 for years if such restrictions are put in place.

There is a difference between a model that achieves "fair use" of copyrighted work and one that regurgitates copyrighted work without attribution.

Re: All public GitHub code was used in training Copilot

#80

Post by ghaff »

Tenoke wrote: post #74
mikehearn wrote: post #66

ML novice question: is this atypical when training models? Wasn't GPT-3 trained on a lot of copyrighted data? My gut instinct, which is based on very low-information, is that it would be pretty hard to train models if you could only use open-licensed material.

Yes, it would stiffle NLP research immensely and we likely wouldn't see anything better than gpt3 for years if such restrictions are put in place.

You're basically seeing how some people would have had open source play out. You can look at and use the code but not to make money or in any other way that I personally disapprove of. This is a world where open source would have ended up being pretty much irrelevant.
Post reply on HN