Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

Post reply on HN
Print view View original

11–20 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#11

Post by Gratsby »

From CocoaPods.org:

> CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. It has over ten thousand libraries and can help you scale your projects elegantly.

The developer response:

> [As CocoaPods developers] Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (users in all time zones) or, frankly, at all. Trying to have a few devs do this, possibly in their spare-time, is a sure way to burn them out. And then there’s also the funding aspect to such a service.

--

So they want to be the go-to scaling solution, but they don't want to have to spend any time thinking about how to scale anything. It should just happen. Other people have free scalable services, they should just hand over their resources.

Thank goodness Github thought about these kinds of cases from the beginning and instituted automatic rate limiting. Having an entire end user base use git to sync up a 16K+ directory tree is not a good idea in the first place. The developers should have long since been thinking about a more efficient solution.

Re: CocoaPods downloads max out five GitHub server CPUs

#12

Post by zeveb »

revelation wrote: post #7
sdegutis wrote: post #5

I love how this was like the perfect storm of things that could go wrong, and how it seems like mhagger is just amazed more than anything else.

Hardly. Do they hit edge cases? Yes! Is using GitHub for your CDN a dick move and going to cause problems, regardless? Yes! The response from mhagger is unnecessarily apologetic, and I predict we'll see an official update from GitHub on this soon.

> Is using GitHub for your CDN a dick move and going to cause problems, regardless? Yes!

I don't know about that. Both oh-my-zsh[1] and emacs prelude[2] use git repos as their code distribution mechanisms, and that works really well. I think the real issues here are exactly what is called out in the issue: poor usage of git, and poor directory layout.

[1] https://github.com/robbyrussell/oh-my-zsh [2] https://github.com/bbatsov/prelude

Re: CocoaPods downloads max out five GitHub server CPUs

#13

Post by Anderkent »

zymhan wrote: post #10

I've always found Github's business model interesting. What if a massive open-source organization (e.g. Fedora, Apache) decided to use it for all of their development, integrating it with continuous builds and all the associated pulls. Of course this isn't likely to happen for a number of reasons, but there are large open source projects that could put a significant load on their infrastructure if they chose to use G…

Presumably they'd get load-limited to the same degree and eventually be forced to move to something they control.

Re: CocoaPods downloads max out five GitHub server CPUs

#14

Post by pjc50 »

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...

"Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)"

In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/CocoaPods/Specs/tree/master/Specs ) which every user downloads.

Package management and sync seems to suffer really badly from NIH. Dpkg is over 20 years old and yum is over a decade old. What's up with this particular wheel that people keep reinventing it seemingly without improvement?

Re: CocoaPods downloads max out five GitHub server CPUs

#15

Post by pavlov »

I've never really understood CocoaPods. Dragging a framework into Xcode was never much trouble, and the amount of 3rd party libraries in a OS X / iOS project ought to be fairly small, so the gains are trivial.

The potential downsides seem much more annoying. Do you really want to have your dependencies on an overloaded central server somewhere?

Re: CocoaPods downloads max out five GitHub server CPUs

#16

Post by athenot »

zymhan wrote: post #10

I've always found Github's business model interesting. What if a massive open-source organization (e.g. Fedora, Apache) decided to use it for all of their development, integrating it with continuous builds and all the associated pulls. Of course this isn't likely to happen for a number of reasons, but there are large open source projects that could put a significant load on their infrastructure if they chose to use G…

It's the same business model that Jira used in the past, when the alternatives were Mantis, Trac and Bugzilla. They had[1] one of the better and well designed issue trackers, free for OSS projects. That turned into a great way to champion adoption within paying customers' organizations.

[1] In my opinion, they have since lost that edge on the UI.

Re: CocoaPods downloads max out five GitHub server CPUs

#17

Post by joshribakoff »

justinclift wrote: post #8
jdcarter wrote: post #3

Wow, really impressive response from GitHub. The right amount of technical detail coupled with balanced tone--halfway between "we support you" and "you make us crazy." One correction to the post title: it's not maxing five nodes, but five CPUs.

Yeah, 5 cpu's is an order of magnitude difference. ;)

Wait, so the CPU isn't the big white tower sitting under my desk?!

Re: CocoaPods downloads max out five GitHub server CPUs

#19

Post by geofft »

Gratsby wrote: post #11

From CocoaPods.org: > CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. It has over ten thousand libraries and can help you scale your projects elegantly. The developer response: > [As CocoaPods developers] Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (user…

I think the word "scale" is used to mean different things in the two sentences you quoted. In the first case, "scale" is about package management and dependency tracking, helping an individual software project approach large scale (many third-party dependencies with possibly conflicting requirements, new developers who need to get up-to-speed quickly, etc.). In the second case, "scaling" is about distribution of the CocoaPods metadata to large numbers of users, each with their own (possibly small) software project.

Sentence 1 would still be true if CocoaPods was only used by ten companies developing the ten biggest (in terms of lines of code) Objective-C projects, but there would no longer be a need to scale in the sense of sentence 2.

Re: CocoaPods downloads max out five GitHub server CPUs

#20

Post by arrakeen »

dankohn1 wrote: post #9
onli wrote: post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

Yes, it's a hugely positive advertisement for the essential role that Github plays in the open source community. Also impressive is the followup message from a Github API developer (and Homebrew maintainer) offering access to a beta API that Homebrew is working with to reduce load. https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...

taking a look at homebrew's implementation of this new API feature, i fail to see how it would dramatically reduce fetches for their (homebrew's) use case. from what i understand, it will only be called when the user manually invokes `brew update`. how often are users calling this command over and over?

that being said, i do believe it could help cocoapod's use case since the fetches are done automatically (as i understand it)

Post reply on HN