Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

61–70 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#61
post #42
post #26

Earlier quoted context omitted.

It's very "sharing economy": someone else has a resource you can use for free, so why not take it? (Edit: )

Because abusing that resource makes sure you (or anyone else) won't be able to take it in the future. I thought that's pretty obvious?

pjc50's question was rhetorical, and was offering a critique of the "sharing economy" mindset. That it should be pretty obvious is exactly the point.

Re: CocoaPods downloads max out five GitHub server CPUs

#62
post #50

I find it amusing how GitHub's contact[1] form has (probably a recent addition): > GitHub Support is unable to help with issues specific to CocoaPods/CocoaPods. --- [1]: https://github.com/contact

It's pretty nifty - it seems to pull in from your history, and then show `user/repo` for the most recent one you've looked at.

Re: CocoaPods downloads max out five GitHub server CPUs

#63
post #54

I love GitHub's response, but I would urge the project more strongly to use modern CDN solutions. CDNs are dirt cheap and incredibly powerful nowadays, for the data sizes that we're talking about here.

How would you define "dirt cheap"? One of the most popular CDN's out there (Akamai) charges $3,500/month for 10TB/month. Who's going to foot that bill? :)

Re: CocoaPods downloads max out five GitHub server CPUs

#65

Earlier quoted context omitted.

Using GitHub for a CDN would be hotlinking to a .js file from your production website, not whats happening here. Other package managers do the same, for example "bower" will clone from github, it just does normal clones.

I think you're missing my distinction here. Yes, bower will clone from GitHub, but of course only the packages you actually use; in the same sense that Go can import source from GitHub repos. These repos are storing code and using GitHub as intended. But CocoaPods/Specs is the equivalent of someone collecting all possible packages you could ever use in Bower in one big GitHub repo.

It's only the package metadata (the spec) though, same as Homebrew, it's not like the package code is pushed to the repo.

Though I wonder if the actual project-fetching is similarly daft.

Re: CocoaPods downloads max out five GitHub server CPUs

#66
post #15

I've never really understood CocoaPods. Dragging a framework into Xcode was never much trouble, and the amount of 3rd party libraries in a OS X / iOS project ought to be fairly small, so the gains are trivial. The potential downsides seem much more annoying. Do you really want to have your dependencies on an overloaded central server somewhere?

Until recently iOS only supported statically linked libraries which could lead to issues if you needed to use multiple components that had a shared dependency that you needed to upgrade for one reason or another. You couldn't touch the embedded version. Additionally there was no native package manager which made sharing libraries a clumsy affair. Cocoapods makes both cases easier.

Re: CocoaPods downloads max out five GitHub server CPUs

#67
post #50

I find it amusing how GitHub's contact[1] form has (probably a recent addition): > GitHub Support is unable to help with issues specific to CocoaPods/CocoaPods. --- [1]: https://github.com/contact

Looks like this shows the last project you looked at before heading to the contact page. I refreshed the leaf repo and then refreshed the contact page and it mentioned the leaf project at the top.

Re: CocoaPods downloads max out five GitHub server CPUs

#68
post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

[deleted]

Re: CocoaPods downloads max out five GitHub server CPUs

#69
I help run Mozilla's version control infrastructure and the problems described by the GitHub engineer have been known to me for years. Concerns over scaling Git servers are one of the reasons I am extremely reluctant to see Mozilla support a high volume Git server to support Firefox development.

Fortunately for us, Firefox is canonically hosted in Mercurial. So, I implemented support in Mercurial for transparently cloning from server-advertised pre-generated static files. For hg.mozilla.org, we're serving >1TB/day from a CDN. Our server CPU load has fallen off a cliff, allowing us to scale hg.mozilla.org cheaply. Additionally, consumers around the globe now clone faster and more reliably since they are using a global CDN instead of hitting servers on the USA west coast!

If you have Mercurial 3.7 installed, `hg clone https://hg.mozilla.org/mozilla-central` will automatically clone from a CDN and our servers will incur maybe 5s of CPU time to service that clone. Before, they were taking minutes of CPU time to repackage server data in an optimal format for the client (very similar to the repack operation that Git servers perform).

More technical details and instructions on deploying this are documented in Mercurial itself: https://selenic.com/repo/hg/file/9974b8236cac/hgext/clonebun.... You can see a list of Mozilla's advertised bundles at https://hg.cdn.mozilla.net/ and what a manifest looks like on the server at https://hg.mozilla.org/mozilla-central?cmd=clonebundles.

A number of months ago I saw talk on the Git mailing list about implementing a similar feature (which would likely save GitHub in this scenario). But I don't believe it has manifested into patches. Hopefully GitHub (or any large Git hosting provider) realizes the benefits of this feature and implements it.

Post reply on HN