Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

171–180 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#171
I have been seeing this trend of GitHub getting "abused" for purposes other than hosting source code.

- My school uses GitHub to host and track our software engineering project (which still can be argued as OSS).

- People using GitHub issue system as a forum.

- Friends uploading pdfs to GitHub.

- Recently people posted on HN about using GitHub to generate a status page.

I think this is a really bad trend and people should stop doing that.

Re: CocoaPods downloads max out five GitHub server CPUs

#172

Earlier quoted context omitted.

This may very well be the first time that Cocoa Pods was told that they're consuming a huge amount of resources. They might not have even known that what they're doing is considered abuse.

hard for me to believe no one was ever curious enough to do the math

People are starting to get used to the idea that you hit someones api and they scale for you. I mean from a company based on scaling that's strange but there's a lot of 'magic' that goes into a lot of providers being able to always respond to an API call and most don't see it.

Re: CocoaPods downloads max out five GitHub server CPUs

#173
post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

Agreed. Perhaps better even would be an automatic message that says that rate-limiting is in effect, explaining the reasons.

Re: CocoaPods downloads max out five GitHub server CPUs

#174
post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

Debian's sync may be nicer, but their client-side solution leaves a bit to be desired. Trivial apt operations (e.g. trying to install a package which is already installed) on an NSLU2 (an ancient 266MHz ARM machine) take several minutes, whereas the same operation takes several seconds on a modern laptop. It turns out this is due to the fact that Debian "main" (Packages.gz) has ballooned to 32MB of plain text when un…

> A side project I've started looking into...

Wouldn't cleaning up the package search interface be a similar effort with much greater payoff?

Re: CocoaPods downloads max out five GitHub server CPUs

#175
post #96

Earlier quoted context omitted.

How would you define "dirt cheap"? One of the most popular CDN's out there (Akamai) charges $3,500/month for 10TB/month. Who's going to foot that bill? :)

That's a list price that they charge those who don't care to negotiate. CloudFlare starts at $0 and doesn't meter/charge for bandwidth. CloudFront charges 9 cents per GB and is integrated with other AWS APIs (which can be very useful). Both those solutions could be managed with a donation pool, I would try the CloudFlare free tier first.

CloudFlare's terms of service specifically forbid using it as a file hosting service.

Re: CocoaPods downloads max out five GitHub server CPUs

#176

Earlier quoted context omitted.

Dynamic libraries don't help shared dependencies, do they? In both cases, you want the shared dependency to be in its own library. There is a difference in that duplicated dependencies in shared libraries will produce a subtle runtime error rather than an obvious build-time error, but that's not really better.

For the sake of brevity I didn't dive as deeply into the subject as I maybe should have, but the solution you get with Cocoapods to this is if you have two Pods that point at a shared dependency in another Pod you end up with just one copy. I forget how they handle version pinning as it's been a little while since I've had to actually worry about it (I do little iOS work these days) but I seem to remember that in thi…

I'm sure CocoaPods handles it, but what I'm wondering about is how static versus dynamic libraries come into the picture. As far as I can tell, that has no relevance to the problem of shared dependencies.

Re: CocoaPods downloads max out five GitHub server CPUs

#177

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

I had the same impression from alloy's response. I've basically read it as "hi ho we will not change anything". And it had this passive agressive ring to it, with the hand clapping and the hurray in the beginning and the stone walling in effect.

That was the smuggest use of emoji I've seen in a while

Re: CocoaPods downloads max out five GitHub server CPUs

#178

Earlier quoted context omitted.

This may very well be the first time that Cocoa Pods was told that they're consuming a huge amount of resources. They might not have even known that what they're doing is considered abuse.

hard for me to believe no one was ever curious enough to do the math

Based on just looking at how some of my employer's customers use our service, plenty of them are completely clueless that they're well outside of normal usage patterns.

Re: CocoaPods downloads max out five GitHub server CPUs

#179

Earlier quoted context omitted.

taking a look at homebrew's implementation of this new API feature, i fail to see how it would dramatically reduce fetches for their (homebrew's) use case. from what i understand, it will only be called when the user manually invokes `brew update`. how often are users calling this command over and over? that being said, i do believe it could help cocoapod's use case since the fetches are done automatically (as i unde…

Hello! I'm the Homebrew maintainer and GitHub employee who wrote this. The main thing this API does for Homebrew is make no-ops really fast for `brew update`. As you point out this results in no speedup (in fact a tiny slowdown) for the case where you only run it where you know you have changes. Where it becomes useful is if you are using multiple taps (Homebrew's 3rd-party repositories) which update infrequently or…

If you plan to rehaul Homebrew, I suggest having by default 3 operations, "update", "fetch" and "install" because some people might find themselves in the situation of having bad connectivity(especially low bandwidth) and being able to fetch the sources, to compile later is very important. Especially having "install" issue synchronous "update" operations is bad if you're on a network with high packet loss, like tethering to one's phone during vacation, etc... Of course, that requires being able to have repeatable "fetch" operations, and putting a local cache between the "fetch" and the "install" operation, so that if a "fetch" succeeds, a later "install" will not fail with "file not found". I've never used Homebrew, but that's advice from having used many package managers on Linux and other *nixen. My apologies if it's redundant.

Re: CocoaPods downloads max out five GitHub server CPUs

#180

Earlier quoted context omitted.

You're quibbling semantics. Using a GitHub repo as a high traffic code CDN and keeping 5+ cores pegged while being the single biggest consumer of resources across the whole platform could be reasonably defined as an abuse of the service. Primary definitions on Google: "abuse" verb: use (something) to bad effect "abuse" noun: the improper use of something

Dabbling in the 'kind of customer we can afford to lose' territory.

Not really. Imagine the backlash github would receive here.
Post reply on HN