Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

thehftguy.com

Post reply on HN
Print view View original

21–30 of 127 posts

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#21

Post by marginalia_nu »

mewpmewp2 wrote: post #20
marginalia_nu wrote: post #5

Earlier quoted context omitted.

> It's almost trivial to built a bot to scrape OS code from anywhere on the web. Seems like a logistical nightmare to me. Git repos interact spectacularly poorly with web scraping in general.

I would've said you should download only archives, but really I think commits are also very important data since that shows the actual changes in the code which would be very useful to train AI to suggest changes to the code.

There are valid non-evil reasons for git hosts to want to throttle and put up obstacles toward scraping as well, both via crawlers or 'git clone' or whatever. These are very expensive operations.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#22

Post by flockonus »

marginalia_nu wrote: post #5
samwillis wrote: post #4

> Microsoft GitHub is the largest collection of open source code in the world. Microsoft GitHub is in a unique and dominant positions to host and access and distribute most of the open-source code in the world No, it's not in a "unique and dominant position". Open source code is freely available online, it's almost trivial to build a bot to scrape OS code from anywhere on the web (GitHub included). The comparison to…

> It's almost trivial to built a bot to scrape OS code from anywhere on the web. Seems like a logistical nightmare to me. Git repos interact spectacularly poorly with web scraping in general.

It appears to be the exact opposite to me, `git clone --depth 1 ...` will give you a code that you can know exactly how to parse, vs. webpages that have all sorts of semantical issues.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#23

Post by williamcotton »

NoZebra120vClip wrote: post #9
h2odragon wrote: post #2

When I published stuff to GitHub, it had open licenses: i wanted anyone and everyone to make whatever use of it they could. I didn't foresee this use, and I'm not fond of Microsoft (to say the least); but it certainly falls into the area of things I explicitly allowed when publishing. I suspect many others who publish there feel the same way.

Perhaps your license permits CoPilot reuse, but that is not every F/OSS license. There are some which require attribution of the original authors. There are some which require the distributor to make available any source code, and any modifications made to the software. Software authors are not upset about the mere reuse of their code, it's the violations of such license terms that are problematic. If attribution is…

First, if the copying is found to be fair use (which is very likely), then attribution or other requirements of a copyright license will not be required.

Second, the only aspects of code that needs to follow the license are the parts of the code that are covered by copyright. That excludes anything that is functional. Since optimizations are functional and not expressive in nature then, for example, an optimized sorting algorithm would not be covered. What would be covered is how that algorithm is organized… the API, file structure, class names, ie, the arbitrary parts of code that everyone argues about.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#24

Post by marginalia_nu »

flockonus wrote: post #22
marginalia_nu wrote: post #5

Earlier quoted context omitted.

> It's almost trivial to built a bot to scrape OS code from anywhere on the web. Seems like a logistical nightmare to me. Git repos interact spectacularly poorly with web scraping in general.

It appears to be the exact opposite to me, `git clone --depth 1 ...` will give you a code that you can know exactly how to parse, vs. webpages that have all sorts of semantical issues.

Git clone is a very expensive operation. Git hosts generally will try to prohibit mass git clone:ing for this reason.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#25

Post by eternalban »

samwillis wrote: post #4

> Microsoft GitHub is the largest collection of open source code in the world. Microsoft GitHub is in a unique and dominant positions to host and access and distribute most of the open-source code in the world No, it's not in a "unique and dominant position". Open source code is freely available online, it's almost trivial to build a bot to scrape OS code from anywhere on the web (GitHub included). The comparison to…

> The comparison to the Google Books antitrust falls down completely, Google had a dominant position because it had the resources to scan all books. Anyone can build a collection of almost all open source code.

Copying a file is not the same thing as "scanning" a book. To scan you first need to get your hands on the book (the download part) and then use industrial scanners to scan them. So apple-apple comparison here is scanning training & scanned collection of books trained model, and finally the portals to the loot: Google Books Github+VSC.

Not everyone has the resources to actually process -- that is train the 'model' -- using the publicly available 'data'. Most also don't also own Github and VSC platforms to field their model. In fact, is anyone other than microsoft in a position to both scrape OSS, train a coding AI, and then include that tool in dominant software development platforms?

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#26

Post by mtlynch »

>Open source code on GitHub might be thought of as “open and freely accessible” but it is not. It’s possible for any person to access and download one single repo from GitHub. It’s not possible for a person to download all repos from Github or a percentage of all repos, they will hit limitations and restrictions when trying to download too many repos. (Unless there’s some special archives or mechanisms I am not aware of).

There actually is a convenient archive for accessing GitHub-hosted code in bulk. All GitHub source code is available for bulk analysis in Google BigQuery.

https://cloud.google.com/blog/topics/public-datasets/github-...

I still don't support GitHub training Copilot on other people's code without permission, but this particular part of OP's argument is incorrect.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#27

Post by bilqis »

samwillis wrote: post #4

> Microsoft GitHub is the largest collection of open source code in the world. Microsoft GitHub is in a unique and dominant positions to host and access and distribute most of the open-source code in the world No, it's not in a "unique and dominant position". Open source code is freely available online, it's almost trivial to build a bot to scrape OS code from anywhere on the web (GitHub included). The comparison to…

Github also has access to private repositories.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#28

Post by hanselot »

williamcotton wrote: post #18
jackdaniel wrote: post #8

Earlier quoted context omitted.

This is addressed in the same paragraph - you can't scan/download "whole" github because you'll be throttled.

There’s no need to crawl for your own dataset: https://pile.eleuther.ai/

@article{pile, title={The {P}ile: An 800GB Dataset of Diverse Text for Language Modeling}, author={Gao, Leo and Biderman, Stella and Black, Sid and Golding, Laurence and Hoppe, Travis and Foster, Charles and Phang, Jason and He, Horace and Thite, Anish and Nabeshima, Noa and Presser, Shawn and Leahy, Connor}, journal={arXiv preprint arXiv:2101.00027}, year={2020} }

So if I understand this correctly, the Pile is for code from 2020 backwards? If I wanted anything released in the past 3 years, say something in the SOTA AI space (where a month is a lifetime), I would need the scraper again?

I don't follow how this can compare to direct, live, unrestricted access. I suppose this is just my own hatred of Microsoft shining through. Of course we should accept the status quo, because how dare we suggest Microsoft could operate in a manner that is anti-competitive.

For anyone else trying to catch up, just rent a datacenter, write a crawler, deal with all the intricacies of keeping it in sync in real-time. This sounds trivial, simple even.

I wonder why nobody is doing it? Perhaps everyone doesn't have access to petabytes of storage space, unlimited bandwidth, unlimited proxy-jumps etc.

So the alternative is to buy github?

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#29

Post by tpxl »

19h wrote: post #11

What a ridiculous article. Copilot does not violate antitrust law. GitHub is not a monopoly just because open source devs choose to host there. Devs are free to use GitLab or whatever. Comparing this to Google Books is silly. Google stole copyrighted books. Copilot uses freely shared open source code. No copyright issue. The article claims "Open source code on GitHub might be thought of as 'open and freely accessible…

> The MIT and Apache licenses explicitly allow reuse

> No evidence Copilot [...] violated any licenses

Both of these allow redistribution _if you include the license_. Copilot doesn't include any licenses in the code it distributes. You can argue whether that's fair use or not, but you can't argue that it doesn't respect the license.

Re: Why are there no antitrust claims vs. GitHub Copilot, when there is a precedent?

#30

Post by williamcotton »

shanebellone wrote: post #19
FeepingCreature wrote: post #7

Earlier quoted context omitted.

Wow, that's horrible. I didn't know that sampling and mashups required an explicit license. Looks like it's in the EU as well. edit: Hm, Pelham v Hütter C-476/17 might offer some grace for mashups under the quotation exemption at least. Though I wouldn't rely on that.

"Wow, that's horrible. I didn't know that sampling and mashups required an explicit license." The conclusion essentially boils down to "remixing is not fair use". Today's hip-hop is a direct result of that decision because sampling became prohibitively expensive.

It’s not legally fair use but no musicians consider it stealing.

Remember, the session players who wrote those hooks and grooves were not given a copyright. They got paid a flat fee. They could not care less if their drum beat, bass line or horn part was reused in a creative new way. The lawyers of the copyright holders sure do care, though!

Post reply on HN