Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

191–200 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#191

Earlier quoted context omitted.

Dabbling in the 'kind of customer we can afford to lose' territory.

Not really. Imagine the backlash github would receive here.

As an ops person, I can tell you that I would probably upgrade my paid github account instead of cancel it if a project was thrown out that had the audacity to issue the statement that they don't care about the infrastructure they use for free so they can focus on the important things - namely developer funding.

Re: CocoaPods downloads max out five GitHub server CPUs

#192

The criticism against CocoaPods here seems awfully harsh. Think about it from their perspective. GitHub advertises a free service, and encourages using it. Partly it's free because it's a loss leader for their paid offerings, and partly it's free because free usage is effectively advertising GitHub. CocoaPods builds builds their project on this free service, and everything is fine for years. Then one day things start…

I entirely agree with this. GitHub gets so much advertising + community from open source projects like this.

Also, I'm amazed this is even a problem. 5 CPUs is not a lot in the scheme of things (even if they mean physical instead of cores). TBs of bandwidth are also virtually free compared to a company the size of Github.

Even better: they are getting basically real world loadtested for free and finding loads of pain points, which may hit paying customers.

Unless I'm missing something, fire more metal at the problem. Many companies would love to be able to have every single cocoapod user (which is nearly every iOS developer) have to type github.com into their terminal for the cost of a bunch of servers + some bandwidth.

Pretty strange, unless this is hitting some really bad area of their service that can't easily be scaled out of (but i would be surprised)

Re: CocoaPods downloads max out five GitHub server CPUs

#194
post #188

Earlier quoted context omitted.

> EDIT: the upside is that cocoapods will have to either rethink there architecture in order to eat less resources or move to their own paid infrastructure because their package manager will soon be less than functional given the aggressive rate limiting github is performing. I'd like to see both happen: * CocoaPods refactoring to be more efficient * GitHub providing open source projects the option to buy reserved ca…

> GitHub providing open source projects > the option to buy reserved capacity. I have no affiliation with GitHub, but I'd guess that if you were paying for one of their $200/month organization plans[1] you'd be having a very different conversation with them about rate limiting. 1. https://github.com/pricing

I would be interested if any of the top five open source projects consuming the most resources are paying Github anything.

Re: CocoaPods downloads max out five GitHub server CPUs

#195

Only the Apple development community would think it's OK to have 16,000 subdirectories in one place and abuse GitHub as a free CDN instead of putting some actual effort in and develop their own repository infrastructure - you know, like almost every other package manager in existence.

It should be "Apple or Ruby development community", homebrew is using similar tech stack distributing the `Formula`.

Re: CocoaPods downloads max out five GitHub server CPUs

#196
post #188

Earlier quoted context omitted.

> GitHub providing open source projects > the option to buy reserved capacity. I have no affiliation with GitHub, but I'd guess that if you were paying for one of their $200/month organization plans[1] you'd be having a very different conversation with them about rate limiting. 1. https://github.com/pricing

I would be interested if any of the top five open source projects consuming the most resources are paying Github anything.

They probably aren't, but they aren't going to be using the infrastructure in the pathological way CocoaPods is either, which requires you to have a client that uses GitHub on behalf of your users.

I'm just pointing out that the feature you're wishing exists very likely already exists in practice. Unless GitHub is stupid they aren't going to be complaining about you pegging 5 CPU cores for $200/month.

Re: CocoaPods downloads max out five GitHub server CPUs

#197
post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

Care to substantiate that last paragraph? Are you really suggesting OS X users use yum?

It's absolutely possible to install yum on OS X (I know this from experience)

Re: CocoaPods downloads max out five GitHub server CPUs

#198

I have been seeing this trend of GitHub getting "abused" for purposes other than hosting source code. - My school uses GitHub to host and track our software engineering project (which still can be argued as OSS). - People using GitHub issue system as a forum. - Friends uploading pdfs to GitHub. - Recently people posted on HN about using GitHub to generate a status page. I think this is a really bad trend and people s…

The school example and friends uploading pdfs to GitHub are both uses that GitHub encourages. Using GitHub Issues as a forum and a source for generating status pages are both ok from a use/abuse perspective, but you may not have the best experience since that isn't what Issues is intended for.

The issue tracker on new repos has a 'question' tag by default, so Github are gently encouraging using issues as a forum. Though my inner cynic says that makes sense for them - issues tie a project to Github more than the git repo itself does.

It should be fine to come up with new ways to use Github, as long as it's not causing excessive load.

Re: CocoaPods downloads max out five GitHub server CPUs

#199
post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

Debian's sync may be nicer, but their client-side solution leaves a bit to be desired. Trivial apt operations (e.g. trying to install a package which is already installed) on an NSLU2 (an ancient 266MHz ARM machine) take several minutes, whereas the same operation takes several seconds on a modern laptop. It turns out this is due to the fact that Debian "main" (Packages.gz) has ballooned to 32MB of plain text when un…

> Trivial apt operations (e.g. trying to install a package which is already installed) on an NSLU2 (an ancient 266MHz ARM machine) take several minutes

Actually, I'm surprised one can actually run a modern Linux on the NSLU2 given its shameful lack of RAM and slooow USB port. But it was a nice gadget when it came out and it was fun to experiment with it.

> It turns out this is due to the fact that Debian "main" (Packages.gz) has ballooned to 32MB of plain text when uncompressed, comprising more than 41,000 packages, and it has to be parsed and assembled into a dependency tree for every apt operation. This problem screams for SQLite.

Correct me if I'm wrong, but isn't apt (and dpkg) basically composed out of a ton of different (perl/shellscript) modules? So it should be possible to create an interface-compatible sqlite data store.

Re: CocoaPods downloads max out five GitHub server CPUs

#200

Why aren't the packages distributed? Composer is incredibly distributed and likely doesn't cause nearly the same headaches for GitHub. Seems like a poor design decision on the CocoaPods side.

What package? My project may have a dependency on a specific commit that's between point releases.

The problem is not packages, it's the index, containing 16000 subdirectories,

Post reply on HN