Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub’s Large File Storage is no panacea for Open Source

medium.com

Post reply on HN
Print view View original

11–20 of 66 posts

Re: GitHub’s Large File Storage is no panacea for Open Source

#11

Post by jumpwah »

Jasper_ wrote: post #6
paulddraper wrote: post #2

> Case in point: if a very popular Github repository (such as the one for the Linux kernel) decided to start using LFS for some of their files, they would instantly alienate all of their users. They would no longer be able to properly fork the project, or even clone it to get its binary files stored via LFS. Nobody would be able to send a pull request to Linus as a result without considerable effort. Odd example. Lin…

They still use pull requests, though, just in the form of email-based ones. https://git-scm.com/docs/git-request-pull

Except that's not a "pull request".

Re: GitHub’s Large File Storage is no panacea for Open Source

#12

Post by eridius »

paulddraper wrote: post #2

> Case in point: if a very popular Github repository (such as the one for the Linux kernel) decided to start using LFS for some of their files, they would instantly alienate all of their users. They would no longer be able to properly fork the project, or even clone it to get its binary files stored via LFS. Nobody would be able to send a pull request to Linus as a result without considerable effort. Odd example. Lin…

I would imagine that there's a number of people out there who still fork the GitHub mirror of Linux and use that to build their pull requests (which are then submitted via email rather than over GitHub, but which would presumably have the exact same issues).

In fact, there are currently 10,532 forks according to GitHub.

Re: GitHub’s Large File Storage is no panacea for Open Source

#13

Post by eshamow »

I'm not sure I understand why artifacts can't be stored in a different service - even an S3 bucket, if not a real repository service - and fetched dynamically via a build process.

Is there a reason why binary blobs need to be stored directly next to code in order to be versioned?

Re: GitHub’s Large File Storage is no panacea for Open Source

#14

Post by nkurz »

This seems like an odd problem, but I'm not as familiar with Git as I should be. Is there not a reasonable way to download only the most recent version of these large binary files on the initial request, and then download the historical versions only in the (likely very rare) case that the user actually wants to use them? This would seem more useful in this case than hoping that binary diffs the repository small enough.

Re: GitHub’s Large File Storage is no panacea for Open Source

#15

Post by SwellJoe »

dantiberian wrote: post #7

There's a lot of assumptions here about GitHub being greedy. I've got no idea how much money it costs GitHub to support Open Source projects, but it must easily be in the millions. I think that by this point GitHub deserves the benefit of the doubt before launching into vicious accusations.

While I don't think github is deserving of "vicious accusations", I do believe it is foolish to assume that the github we know today will be the github of tomorrow.

SourceForge.net was once an excellent and trustworthy steward of Open Source software projects. It was predicted by some folks in the free software community that it would not always be the case, and alternatives like Savannah were maintained in order to act as a hedge against that concern. I believe it is more than reasonable to assume that github will change, and it would be downright dangerous to assume that we can rely on a profit-motivated corporation (even one as cool as github currently is) to remain a trustworthy repository forever.

So, sure, say nice things about github; I also think github is a good product, and I appreciate their free hosting for OSS projects. And, sure, you should use github if it provides value for you and you're willing to accept the price. But, don't ask me to trust they'll never change, because history indicates they will. It's probably also unfair to suggest that someone criticizing some valid concerns about github's current behavior, based on their own experience with Open Source projects hosted at github, are making "vicious accusations".

Re: GitHub’s Large File Storage is no panacea for Open Source

#16

Post by adrusi »

nkurz wrote: post #14

This seems like an odd problem, but I'm not as familiar with Git as I should be. Is there not a reasonable way to download only the most recent version of these large binary files on the initial request, and then download the historical versions only in the (likely very rare) case that the user actually wants to use them? This would seem more useful in this case than hoping that binary diffs the repository small enou…

Also not very well versed in git, but my understanding is that there is a way to clone a repo to only include latest revisions, but that this limits usage of git. I believe that fixing this was an area of active development a few months ago, its possible it already landed.

Re: GitHub’s Large File Storage is no panacea for Open Source

#18

Post by Vendan »

jumpwah wrote: post #11
Jasper_ wrote: post #6

Earlier quoted context omitted.

They still use pull requests, though, just in the form of email-based ones. https://git-scm.com/docs/git-request-pull

Except that's not a "pull request".

Except it is. Just cause github does pull requests differently doesn't mean you can't do pull requests in pure git. Remember, git came before github.

Re: GitHub’s Large File Storage is no panacea for Open Source

#19

Post by lemevi »

Edit: I was wrong, however I learned from the conversation so I am leaving it here! Thanks to those who corrected me.

> On the other hand, source files being mostly text, they are more intelligently handled and typically only differences between revisions are stored in the commits.

This is completely incorrect, git stores whole blobs from one commit to the other.

svn stored patches, but git does not. Every version of a file is stored in its entirety in your git tree since the beginning of the repository's existence. This is one of the reasons why git is so fast. You can go through your objects in your .git directory and verify this for yourself[0].

    $ find .git/objects -type f
      .git/objects/ff/a5d733354ae6f8bdc67764d58d87c9a3161f66
      .git/objects/ff/deb08f4856bd6eb5b31d7f800b3e480ae3e2e0
    $ git cat-file -p ffa5d733354ae6f8bdc67764d58d87c9a3161f66
    ...file contents appear...
[0] https://git-scm.com/book/en/v2/Git-Internals-Git-Objects

Re: GitHub’s Large File Storage is no panacea for Open Source

#20

Post by pbiggar »

How uncharitable can a single blog post be! The entire post is discredited by the author repeatedly projecting his unfounded opinions onto GitHub, such as

"My guess is that some high-level greedy marketing dickwad, completely unaware of the asinine implications of his brilliant idea, signed off on this dumb-as-a-bag-of-rocks pricing model."

"All the marketing material pimping GitHub’s LFS support [...]. I do not believe this is unintentional."

"This is completely batshit. The side effect of this pernicious, greedy pricing model is to [...]"

"I honestly couldn’t believe that GitHub would be willing to do something that shortsighted, visibly motivated by greed from the cash they thought they could extract from some of their users".

Charitable explanation for forks not working: they haven't yet written the code to make this work with forks, and it's better to ship something working early, than to make it work in all cases.

Charitable explanation for charging for bandwith: bandwidth costs money. (I believe this is a real problem for Dropbox, which doesn't charge for bandwidth but must still pay for it). Also, all CDNs, and also AWS charge for bandwidth.

Overall, while GitHub may be able to support it's OSS folks better by changing the pricing on some parts of its product, this post is incredibly uncharitable. I hope the OP will consider removing the unfounded narrative that he's projecting onto GitHub (esp the "marketing dickwad" thing - wtf) and focus on the facts.

[Disclaimer: my company partners with GitHub on lots of stuff]

Post reply on HN