Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

Post reply on HN
Print view View original

151–160 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#151

Post by mordocai »

nostrademons wrote: post #112
protomyth wrote: post #106

Earlier quoted context omitted.

> They aren't really deriving any benefit out of the work they do on CocoaPods No direct financial benefit, but they are deriving a benefit out of their work.

Presumably - otherwise they wouldn't be doing it - but it often doesn't take all that much to flip a volunteer from "Okay, this is cool, I can help other people and learn some stuff as well" to "Fuck this, it's way more trouble than it's worth." Top amongst this is when the people you're helping expect you to give them free work. Jobs compete with other jobs, and most people expect that they'll have to do some unplea…

It also doesn't take much to move a company from "Okay, we'll help you by hosting your shit for free" to "Fuck you, you're banned since you are ungrateful bastards" (in nicer language of course).

Re: CocoaPods downloads max out five GitHub server CPUs

#152

Post by soheil »

I was up until 2am last night trying to publish my Pod [1] and Github kept timing out.

I had no idea it was just CocoaPods repo because my other repos were working fine. I accepted defeat, went to bed and everything was working great in the morning.

[1] https://github.com/soheil/SwiftCSS

Re: CocoaPods downloads max out five GitHub server CPUs

#153

Post by betenoire »

kingnothing wrote: post #57
izacus wrote: post #42

Earlier quoted context omitted.

Because abusing that resource makes sure you (or anyone else) won't be able to take it in the future. I thought that's pretty obvious?

This may very well be the first time that Cocoa Pods was told that they're consuming a huge amount of resources. They might not have even known that what they're doing is considered abuse.

hard for me to believe no one was ever curious enough to do the math

Re: CocoaPods downloads max out five GitHub server CPUs

#154

Post by nicky0 »

rmoriz wrote: post #81

CocoaPods (and Homebrew) mainly exist because of a lack of tooling in the typical Apple ecosystem. So I would blame Apple for not supporting the community with money or tooling. Letting GitHub with its limited amount of funding pay the bill isn't a nice move. Apple dev relations should throw some money at GitHub so they can provide some dedicated resources or offer to pay the cost of other solutions (like a 3rd party…

It's probably worth pointing out here that Apple is currently working on a package manager (https://github.com/apple/swift-package-manager).

Re: CocoaPods downloads max out five GitHub server CPUs

#155

Post by Wintamute »

sanjeetsuhag wrote: post #137
manyxcxi wrote: post #116

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

> you're abusing Cocoapods uses GitHub. No abuse here.

You're quibbling semantics.

Using a GitHub repo as a high traffic code CDN and keeping 5+ cores pegged while being the single biggest consumer of resources across the whole platform could be reasonably defined as an abuse of the service.

Primary definitions on Google:

"abuse" verb: use (something) to bad effect

"abuse" noun: the improper use of something

Re: CocoaPods downloads max out five GitHub server CPUs

#156

Post by dinkumthinkum »

Gratsby wrote: post #11

From CocoaPods.org: > CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. It has over ten thousand libraries and can help you scale your projects elegantly. The developer response: > [As CocoaPods developers] Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (user…

It was pretty shocking to see a couple of the responses. If point is to be a package manager then they should see this as important part of the project and not just see it as Github's duty to provide this free service. Basically they are saying their service cannot exist with Github or someone will to expend the, apparently fairly significant, capital to provide the backend to their service. And this is all happening at a time when many people have figured out how to provide package management services in a reasonable way.

Re: CocoaPods downloads max out five GitHub server CPUs

#157

Post by MBCook »

sanjeetsuhag wrote: post #137
manyxcxi wrote: post #116

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

> you're abusing Cocoapods uses GitHub. No abuse here.

It's very clear GitHub was not designed to be used as some project's personal CDN for this kind of traffic.

It's abuse. Wasn't intentional, but at the scale that Cocoapods is running it's abuse.

Re: CocoaPods downloads max out five GitHub server CPUs

#158

Post by cmyr »

manyxcxi wrote: post #116
onli wrote: post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

I think you could assume better of people. His response was maybe a bit tone deaf, but text is a very poor way of communicating mood and circumstance. There are any number of creditable explanations from tiredness to language skills that could make sense of the response you're offended by.

fwiw: "I would have a really hard time working for someone whose first inclination was always towards criticism over accommodation or compassion." But then I also acknowledge there may be a whole bunch of other stuff going on here behind the scenes. ;)

Re: CocoaPods downloads max out five GitHub server CPUs

#159

Post by web007 »

acdha wrote: post #122
web007 wrote: post #46

I keep coming back to point #4 - who ever thought that 16k objects in a single directory would be a good idea? Ever since FAT that's been a bad idea, and while modern FSes will handle it without completely melting down it's still going to cause long access operations on anything to do with it. Even Finder or `ls` will have trouble with that, and anything with * is almost certainly going to fail. Is the use-case for t…

I do think that your last sentence is the answer: if you're using a package manager instead of working with the directory heavily, this isn't a visible problem which is going to motivate people to work on it. The other side to consider: “one directory per package” is a very simple policy and it feels right in many ways to people (e.g. Homebrew has a similar structure because it's a natural fit for the domain). If the…

One directory per package is completely sensible, just not all in one bunch. It's even fine if the mapping is to a flat namespace at something like the HTTP level - I can mod_rewrite /abcdefg to /a/b/c/abcdefg no problem. My only objection is to file- or directory-level structures that are this flat. I might be mentally deficient, but I can't even process anything that's structured this way.

As loathe as I am to admit anything about Perl is good, CPAN got this right. 161k packages by 12k authors, grouped by A/AU/AUTHOR/Module. That even gives you the added bonus of authorship attribution. Debian splits in a similar way as well, /pool/BRANCH/M/Module/ and even /pool/BRANCH/libM/Module/ as a special case.

Tooling can be considered part of the problem in this case. Because the tooling hides the implementation, nobody (in the project) noticed just how bad it was. I hadn't seen modern FS performance on something of this scale, apparently everything I've worked with has been either much smaller or much larger. Ext4 (and I assume HFS+) is crazy-fast for either `ls -l` or `find` on that repo.

It seems like tooling is part of the solution as well, but from the `git` side. Having "weird" behavior for a tool that's so integral to so many projects scares me a little, but it's awesome that Github has (and uses) enough resources to identify and address such weirdness.

Re: CocoaPods downloads max out five GitHub server CPUs

#160

Post by jlarocco »

sanjeetsuhag wrote: post #137
manyxcxi wrote: post #116

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

> you're abusing Cocoapods uses GitHub. No abuse here.

I totally disagree. The CocoaPods usage model is not at all the expected way to use GitHub. I'm surprised using GitHub that way is even allowed by the TOS. It's very obviously a hack to avoid paying for their own infrastructure. The project representative in the issues thread even admits as much in his response.

Just about every other package manager for every other language (Pip, CPAN, Hackage, etc.) uses its own infrastructure.

Post reply on HN