Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I do not agree with Github's use of copyrighted code as training for Copilot

thelig.ht

Post reply on HN
Print view View original

371–380 of 545 posts

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#371

Post by hackinthebochs »

saurik wrote: post #266
moyix wrote: post #185

Earlier quoted context omitted.

What? I don't think I made any claim of the sort. I'm claiming that it does more than mere regurgitation and has done some amount of abstraction, not that it has human-level understanding. As an example, GPT-3 learned some arithmetic and can solve basic math problems not in its training set. This is beyond pattern matching and replication, IMO. I'm not really sure why we should consider Copilot legally different from…

The argument I was responding to--made by the user crazygringo--was that GPT-3 trained on a model of the Windows source code is fine to use nigh unto indiscriminately, as supposedly Copilot is abstracting knowledge like a human engineer. I argued that it doesn't do that: that GPT-3 is a pattern recognize that not only theoretically just likes to memorize and regurgitate things, it has been shown to in practice. You t…

Statistical modelling doesn't imply that GPT-3 is merely regurgitating. There are regularities among different examples, i.e. abstractions, that can be learned to improve its ability to predict novel inputs. There is certainly a question of how much Copilot is just reproducing input it has seen, but simply noting that its a statistical model doesn't prove the case that all it can do is regurgitate.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#372

Post by lstodd »

dmitriid wrote: post #83
shakow wrote: post #78

Earlier quoted context omitted.

> Copyright has concluded that reading by robots doesn’t count. Until someone trains a DNN to generate Mickey Mouse-like cartoons I assume.

There was a joke that all ML will be immediately banned the moment there's a Copilot for RIAA-licensed songs.

There were at least three PoCs AFAIK. Surely there were more.

Problem is, the results too closely replicate the source, as in "Suck just as much". And thus there is no demand for this kind of thing.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#373

Post by astrange »

thewakalix wrote: post #121
crazygringo wrote: post #111

Earlier quoted context omitted.

Sorry, but you're not recalling correctly. :) The Supreme Court decided Google v. Oracle was fair use. It was 3 months ago: https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_... That's the highest form of precedent, the question has now been effectively settled (unless Congress ever changes the law). Edit: added a dummy hash to end of URL so HN parses it correctly (thanks @thewakalix below)

There seems to be an issue with Hacker News's URL parsing. The final period isn't included as part of the link.

That would be the wrong result in almost all cases.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#374

Post by breck »

tomtheelder wrote: post #314
breck wrote: post #193

It’s time to abolish copyright ( http://www.breckyunits.com/the-intellectual-freedom-amendmen... ). It absolutely makes no sense—unless your rich and don’t care about the progress of the arts and sciences. You can spin your wheels all you want but going from simple first principles it is fundamentally flawed. If you believe ideas can be property, then you believe people can be property.

> If you believe ideas can be property, then you believe people can be property. Can you defend that? I generally think copyright isn't a great idea as it exists, but this statement feels extremely dubious at best .

> Can you defend that?

Yes. Though I'm not the sharpest tool in the shed, so the delivery may be less than ideal.

Copyright gives PersonA legal control over a subset of PersonB's behavior, even when PersonA is not involved. This is hard to defend, unless you are fine with people being property. Slavery gives PersonA legal control over PersonB's behavior. Under Slavery, PersonB has one master with lots of control. Under Copyright, PersonB has lots of masters with small controls.

Is there a way to believe that ideas can be property without it being a system of slavery? There's no logical way to make that work. Think about the moment that copyright "expires". At that instant, does matter disappear? Did property vanish? What changed? The only thing that changed was each person suddenly gained a little more freedom—the ability to share a new sequence that they couldn't share before. The property rights that the copyright holder had over other people went away. People became more free.

Anyone who thinks for themselves should be able to quickly deduce that these laws are shades of slavery laws, and not about property rights. I figured that out before I could legally drink. It's not that complicated. The question is why are so many duped? I think it's probably a question of priorities (I would say Freedom of Speech and the Press are more fundamental, and then Freedom to Remix and Distribute would be next) or perhaps it's because before the Internet there wasn't enough uncontrolled bandwidth for the truth to get out, or perhaps it's that the people are bombarded over and over again by the big lie from the moment of childhood—look at the FBI Warnings at the beginning of Disney Movies, or the dozens of times per day that you see the phrase "All rights reserved".

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#375

Post by stale2002 »

caconym_ wrote: post #243
kingsuper20 wrote: post #213

Earlier quoted context omitted.

>Copyright doesn't just benefit huge corporations. For instance, without it, independent artists who rely on copying for distribution (authors, musicians, etc.) would find it much more difficult to make money off their work, That doesn't look like it's the point to me. ""[the United States Congress shall have power] To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inven…

> To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right , to their respective Writings and Discoveries. What better way to promote said Progress than by making sure said Authors and Inventors can make enough money off their work to keep doing it? As written, it's a roundabout way to get at the instrumentality of capital, but if that's not what t…

> What better way to promote said Progress

There are a lot of better ways. Having more information being public and free, and usable by tools like this sounds like an excellent way of promoting progress.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#376

Post by astrange »

robbedpeter wrote: post #320
saurik wrote: post #87

Earlier quoted context omitted.

This model doesn't learn and abstract: it just pattern matches and replicates; that's why it was shown exactly replicating regions of code--long enough to not be "de minimis" and recognizable enough to include the comments--that happen to be popular... which would be fine, as long as the license on said code were also being replicated. It just isn't reasonable to try to pretend Copilot--or GPT-3 in general--is some k…

Transformers do learn and abstract. Not as well as humans, but for whatever definitive of innovation or creativity you wanna run with, these gpt models have it. It's not magic, it's math, but these programs are approximating the human function of media synthesis across narrowly limited domains. These aren't your crazy uncle's Markov chain chatbots. They're sophisticated bayesian models trained to approximate the func…

Are they Bayesian? It would be good if ML models were Bayesian (they'd be able to show uncertainty better) but they usually aren't.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#377

Post by notsureaboutpg »

TheRealPomax wrote: post #315
jcelerier wrote: post #61

Earlier quoted context omitted.

> As a human, I am allowed to read copyrighted code and learn from it. Of course not. Reading some copyrighted code can have you entirely excluded from some jobs - you can't become a wine contributor if it can be shown you ever read Windows source code and most likely conversely. Likewise, you can't ever write GPL VST 2 audio plug-ins if you ever had access to the official Steinberg VST2 SDK. Etc etc... Did people fo…

I don't believe you on this in the slightest. This sounds like you making up an argument, so cite sources if you want people to believe your claims.

It's very clearly visible on the Wine wiki that people who have ever seen Microsoft Windows source code cannot contribute to Wine due to copyright restrictions:

https://wiki.winehq.org/Developer_FAQ#Who_can.27t_contribute...

I think OP has a point here, personally.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#378

Post by astrange »

jozvolskyef wrote: post #218
saurik wrote: post #139

Earlier quoted context omitted.

So, to verify, your claim is that GPT-3, when trained on a corpus of human text, isn't merely managing to string together a bunch of high-probability sequences of symbol constructs--which is how every article I have ever read on how it functions describes the technology--but is instead managing to build a model of the human world and the mechanism of narration required to describe it, with which it uses to write new…

Imagine for a second a model of the human brain that consists of three parts. 1) a vector of trillion inputs, 2) a black box, and 3) a vector of trillion outputs. At this level of abstraction, the human brain "pattern matches and replicates" just the same, except it is better at it.

Human thought isn't anything like GPT thought - humans can spend a variable amount of time thinking about what to learn from "training data" and can use explicit logic to reason about it. GPT is more like a form of lossy compression than that.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#379

Post by sneak »

CameronNemo wrote: post #340
stingraycharles wrote: post #254

Earlier quoted context omitted.

I’d argue that this new use case is very interesting to open source and how it relates to the various licenses, and not necessarily “the point of open source”. I can imagine people being OK with their code being used as-is, and/or being modified, but not used completely out of context to train some corporate AI to inject code into commercial code based.

Agreed. I am considering relicensing all of my permissively licensed code because of this. The fundamental assumptions I had when releasing that code under a permissive license have been violated.

When you use a permissive license, it’s best you stop thinking of it as your code. You’ve set it free for everyone, and while you may retain copyright in some abstract sense, it really no longer belongs to you.

That’s a good thing.

Re: I do not agree with Github's use of copyrighted code as training for Copilot

#380

Post by ThrowawayR2 »

ImprobableTruth wrote: post #281
ThrowawayR2 wrote: post #229

Earlier quoted context omitted.

It is certainly fascinating to see people start running away from " information wants to be free " and other Free Software principles full tilt when, all of a sudden, it's their livelihoods that are on the line. Unless my recollection is off, the GPL was never the goal of the original Free Software movement; it was merely a tool to get to the end state where all code becomes available for use by anyone for any reason…

Your recollection is off, majorly. I'd recommend looking up the origins of the FSF/GPL/Copyleft. The entire movement essentially got started because Stallman gave Symbolics his (public domain) Lisp interpreter, then Symbolics improved it but refused to share the improvements. "No restrictions" has never been the goal and to claim that they're egoistic hypocrites who are just scared for their own livelihood because of…

> "I'd recommend looking up the origins of the FSF/GPL/Copyleft. "

Are you sure you're in a position to be saying things like that? The closed source Xerox printer driver incident is generally viewed as the origin of RMS's thinking on Free Software, not the Symbolics incident. And, as others have pointed out, you were mistaken even on the particulars of that.

As for Free Software not being about no restrictions, may I remind you of the four freedoms that are at the heart of the Free Software philosophy? Copilot runs afoul of none of them and I would go so far as to say that Copilot is an embodiment of 1 through 3.

"A program is free software if the program's users have the four essential freedoms:

- The freedom to run the program as you wish, for any purpose (freedom 0).

- The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.

- The freedom to redistribute copies so you can help others (freedom 2).

- The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this."

Again, the GPL is a tool to achieve the Free Software philosophy, not the end goal.

Post reply on HN