Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

211–220 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#211
post #200

Why aren't the packages distributed? Composer is incredibly distributed and likely doesn't cause nearly the same headaches for GitHub. Seems like a poor design decision on the CocoaPods side.

What package? My project may have a dependency on a specific commit that's between point releases. The problem is not packages, it's the index, containing 16000 subdirectories,

So the packages themselves aren't part of the repository?

Re: CocoaPods downloads max out five GitHub server CPUs

#212

The criticism against CocoaPods here seems awfully harsh. Think about it from their perspective. GitHub advertises a free service, and encourages using it. Partly it's free because it's a loss leader for their paid offerings, and partly it's free because free usage is effectively advertising GitHub. CocoaPods builds builds their project on this free service, and everything is fine for years. Then one day things start…

I have never seen anywhere that GitHub advertises using them as a CDN.

GitHub is for source control. That means a limited number of people pulling and submitting changes. That does not mean the general public using it as a CDN.

In fact I seem to remember seeing somewhere active discouragement of using it as a CDN.

Re: CocoaPods downloads max out five GitHub server CPUs

#213
post #43

Earlier quoted context omitted.

When a package manager monopolizes that much resources from Github at the expense of others there is no reason to "commit" all resources to this one project. Thus cocoapods getting rate limited because of the obvious bandwidth abuse going on here. mhagger answer is pretty straight forward. EDIT: the upside is that cocoapods will have to either rethink there architecture in order to eat less resources or move to their…

> EDIT: the upside is that cocoapods will have to either rethink there architecture in order to eat less resources or move to their own paid infrastructure because their package manager will soon be less than functional given the aggressive rate limiting github is performing. I'd like to see both happen: * CocoaPods refactoring to be more efficient * GitHub providing open source projects the option to buy reserved ca…

GitHub already has paid accounts. There's nothing I'm aware of that would prevent an open source project from paying for one.

Re: CocoaPods downloads max out five GitHub server CPUs

#214

The criticism against CocoaPods here seems awfully harsh. Think about it from their perspective. GitHub advertises a free service, and encourages using it. Partly it's free because it's a loss leader for their paid offerings, and partly it's free because free usage is effectively advertising GitHub. CocoaPods builds builds their project on this free service, and everything is fine for years. Then one day things start…

I entirely agree with this. GitHub gets so much advertising + community from open source projects like this. Also, I'm amazed this is even a problem. 5 CPUs is not a lot in the scheme of things (even if they mean physical instead of cores). TBs of bandwidth are also virtually free compared to a company the size of Github. Even better: they are getting basically real world loadtested for free and finding loads of pain…

They are not telling them to stop using GitHub, they are giving them advice on making it work better.

Re: CocoaPods downloads max out five GitHub server CPUs

#215

The criticism against CocoaPods here seems awfully harsh. Think about it from their perspective. GitHub advertises a free service, and encourages using it. Partly it's free because it's a loss leader for their paid offerings, and partly it's free because free usage is effectively advertising GitHub. CocoaPods builds builds their project on this free service, and everything is fine for years. Then one day things start…

On the flip side, user 'alloy' gives the response that their decision to use github as a CDN was an explicit decision. In designing a product to scale, they apparently explicitly decided to outsource the 'scaling' part. While it may have been surprising to them, I don't think it should have been so surprising.

> It strikes me as extremely unreasonable to expect a group to avoid abusing a free service when nobody tells them that it's abuse

I don't think so at all. An experienced developer should expect that a free service will rate-limit their offerings at some point, and design around that. Viewing 'free' as 'an eternal resource sponge that we never have to think about' is the extremely unreasonable thing to do, in my opinion. I think that 'abuse' is probably the wrong word to use here, since that implies malice, and they don't appear to be malicious.

Re: CocoaPods downloads max out five GitHub server CPUs

#216

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

I don't know, it sounded to me like he just didn't totally understand what Github was saying. By the end of the thread, it seemed like everyone was agreeing. I wouldn't be comfortable using words like "selfish" to describe any of what I read. I certainly don't think the barb about your willingness to employ people who write things on Github issues threads that you disagree with is helping anyone understand any part o…

> I certainly don't think the barb about your willingness to employ people who write things on Github issues threads that you disagree with is helping anyone understand any part of this situation.

It seems to be one of HN's go-to insults. "Look at this person's behavior, I would never hire them," as if everyone wants to work at your startup.

Re: CocoaPods downloads max out five GitHub server CPUs

#217

Earlier quoted context omitted.

> A side project I've started looking into... Wouldn't cleaning up the package search interface be a similar effort with much greater payoff?

It isn't just package search which is the problem, it is everything which has to parse the packages file, which is basically every apt command. So if you can trim "main" down to 10,000 packages, suddenly every part of apt is faster, and no one has to install any custom apt replacements.

But wouldn't just storing that file as a database solve most of the issues?

Re: CocoaPods downloads max out five GitHub server CPUs

#218

Earlier quoted context omitted.

This is correct. Starting with iOS 8 they finally allowed linking against custom frameworks which is why Carthage is now becoming much more popular. CocoaPods solved a critical problem of getting dependencies linked in, while creating a new problem of having your xcodeproj file and build settings be managed by them. I'll be happy enough to drop them for new projects going forward.

> Starting with iOS 8 they finally allowed linking against custom frameworks I don't keep up with iOS development enough to know if anything has changed with respect to static/dynamic linking in iOS 8, but it has always been possible to use custom frameworks in iOS (eg, frameworks you build yourself, unless the community has another definition for 'custom framework'). The framework directory structure is a bit unorth…

Sorry you're right I should have been more explicit, starting in iOS 8 you can use dynamically linked frameworks.

Re: CocoaPods downloads max out five GitHub server CPUs

#219
post #209
post #179

Earlier quoted context omitted.

If you plan to rehaul Homebrew, I suggest having by default 3 operations, "update", "fetch" and "install" because some people might find themselves in the situation of having bad connectivity(especially low bandwidth) and being able to fetch the sources, to compile later is very important. Especially having "install" issue synchronous "update" operations is bad if you're on a network with high packet loss, like tethe…

I don't get it... If you've never used homebrew, and are only really a user of other package managers, why are you attempting to offer advice on what they've been doing for years. Is this classic "I have an opinion and all of my opinions are great, so I must pontificate!"? Maybe you should just trust that the developers of the only successful package manager for OS X have some idea of what their users want and need..…

Because I'm interested in package managers in general, and I'd like them to improve across operating systems. And I am a seasoned(9 years) veteran of several Linux distributions.

Re: CocoaPods downloads max out five GitHub server CPUs

#220

The criticism against CocoaPods here seems awfully harsh. Think about it from their perspective. GitHub advertises a free service, and encourages using it. Partly it's free because it's a loss leader for their paid offerings, and partly it's free because free usage is effectively advertising GitHub. CocoaPods builds builds their project on this free service, and everything is fine for years. Then one day things start…

I entirely agree with this. GitHub gets so much advertising + community from open source projects like this. Also, I'm amazed this is even a problem. 5 CPUs is not a lot in the scheme of things (even if they mean physical instead of cores). TBs of bandwidth are also virtually free compared to a company the size of Github. Even better: they are getting basically real world loadtested for free and finding loads of pain…

>>Even better: they are getting basically real world loadtested for free and finding loads of pain points, which may hit paying customers.

I think their point is that it's using the system in a way that isn't intended or desired. How does that count as "real world" load testing?

And by that logic, shouldn't anybody who gets hit with a DoS attack just say "thanks"? It's tons of free load testing on your network infrastructure, and you'll definitely find some pain points.

Post reply on HN