Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

221–230 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#221
post #157

Earlier quoted context omitted.

> you're abusing Cocoapods uses GitHub. No abuse here.

It's very clear GitHub was not designed to be used as some project's personal CDN for this kind of traffic. It's abuse. Wasn't intentional, but at the scale that Cocoapods is running it's abuse.

> personal CDN

Discounting the fact that CocoaPods is used by millions of iOS developers...

Re: CocoaPods downloads max out five GitHub server CPUs

#222

Earlier quoted context omitted.

> What if a massive open-source organization (e.g. Fedora, Apache) decided to use it This one is pretty big. https://github.com/torvalds/linux

Sure but it's just the kernel, and that's just a mirror. Linus does not use Github to manage kernel development. In fact he's been vitriolic in the past about how Github does pull requests. I wonder how much traffic the Github Linux repo gets. Seems to me that people who want to use Linux, will go get a distro instead. And people who want to develop the kernel, will follow the kernel development process (which doesn'…

For a period of time when Kernel.org was breached, GitHub was the repository for Linux. [1] I remember reading a review of GitHub by him shortly after. He did not like how Pull Requests or patches worked on GitHub. I'd link it but I'm having trouble finding it.

1. http://www.theregister.co.uk/2011/09/06/linus_torvalds_dumps...

Re: CocoaPods downloads max out five GitHub server CPUs

#223
post #11

From CocoaPods.org: > CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. It has over ten thousand libraries and can help you scale your projects elegantly. The developer response: > [As CocoaPods developers] Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (user…

Well, the flip side is that the CocoaPods developers are all volunteers (right?). They aren't really deriving any benefit out of the work they do on CocoaPods, and if you ask them to take on financial or ongoing maintenance obligations for a volunteer project, they probably just won't do it. The major benefits of CocoaPods existence go to iOS developers, but there's a tragedy of the commons effect here, where no indi…

Ugh, I'm skeptical of giving Apple ownership of any kind of developer tool. We all saw how badly they screwed up TestFlight, and now you want to give them the only OSS package manager?

Re: CocoaPods downloads max out five GitHub server CPUs

#224

Earlier quoted context omitted.

It isn't just package search which is the problem, it is everything which has to parse the packages file, which is basically every apt command. So if you can trim "main" down to 10,000 packages, suddenly every part of apt is faster, and no one has to install any custom apt replacements.

But wouldn't just storing that file as a database solve most of the issues?

Yes, and that would be awesome, but it means rewriting apt.

Re: CocoaPods downloads max out five GitHub server CPUs

#226
post #81

CocoaPods (and Homebrew) mainly exist because of a lack of tooling in the typical Apple ecosystem. So I would blame Apple for not supporting the community with money or tooling. Letting GitHub with its limited amount of funding pay the bill isn't a nice move. Apple dev relations should throw some money at GitHub so they can provide some dedicated resources or offer to pay the cost of other solutions (like a 3rd party…

CocoaPods exists because developers want to learn how to "build apps" but lack the resources to intelligently include and link to 3rd party code in their projects. CocoaPods doesn't enable anything not otherwise configurable via git submodules and Xcode project hierarchies / build settings. Therefore, it's not Apple's problem. In fact, I've talked to a non-trivial amount of engineers (both in Cupertino and long time…

Apple decided to block dylibs from the start of the iOS app store, and I think that friction point from how things are usually done in OSX, C & C++ land before iOS is what started the entire cocapods thing in the first place. The replacement with dynamic frameworks came about 8 years too late in ios 8.

Re: CocoaPods downloads max out five GitHub server CPUs

#227
post #186

Earlier quoted context omitted.

I know you're trying to suggest some good ideas, but never having used homebrew I think you should find out more about homebrew itself before suggesting things. I don't mean you have to be an expert at it, but just to know a bit about how it's used, and who it's used by before trying to suggest things to people. My comment isn't to pass judgement on your suggestion, but if you took a look at homebrew itself you'd be…

It's not always that easy, I've found myself having to go to a place with high-bandwidth internet to fetch stuff, and being able to stay there only a few minutes. Or before boarding a plane(you don't really want to pay for internet on an intercontinental flight), etc... Also, while you might do things as-needed, it's not that difficult to always remember to run an update every time you run brew. And I wasn't even pro…

All I'm saying is that you really need to investigate into the things you are suggesting. Please do the investigation, even if cursory before you jump in.

I am guilty of the same, but in my case, I have an installation I can actually check. You can read the source code if you don't have a mac to work with, but the important thing is this already exists

from the brew man page:

brew(1)

fetch [--force] [-v] [--devel|--HEAD] [--deps] [--build-from-source|--force-bottle] formulae

Download the source packages for the given formulae. For tarballs, also print SHA-1 and SHA-256 checksums.

Re: CocoaPods downloads max out five GitHub server CPUs

#228
post #221
post #157

Earlier quoted context omitted.

It's very clear GitHub was not designed to be used as some project's personal CDN for this kind of traffic. It's abuse. Wasn't intentional, but at the scale that Cocoapods is running it's abuse.

> personal CDN Discounting the fact that CocoaPods is used by millions of iOS developers...

Pretty sure that's the point. If it wasn't used by so many developers, it wouldn't be causing load issues. :)

Re: CocoaPods downloads max out five GitHub server CPUs

#229

Rust's cargo does something similar, however it looks like they were much more conscious of git-scalability (ex: limiting the directories in a single level, only appending lines to files to make diffs small). https://github.com/rust-lang/crates.io-index

For what it's worse, both of these characteristics indeed weren't an accident.

At the time, I wrote a script that hammered git commits into a repository using different strategies and looked at what the git repository would look like after 100,000 and a million commits. The "one version per file, nested in a flat structure" had serious issues.

There may still be scaling limits with the Cargo approach, but if we reach them, we have plans to create a new registry with a new initial commit and let the old registry age out, then rinse/repeat. At the moment, we haven't hit limits yet (with about 1/3 of the packages that Cocoapods has).

Re: CocoaPods downloads max out five GitHub server CPUs

#230

Earlier quoted context omitted.

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

I had the same impression from alloy's response. I've basically read it as "hi ho we will not change anything". And it had this passive agressive ring to it, with the hand clapping and the hurray in the beginning and the stone walling in effect.

Had exactly the same feeling. I felt bad for the GH employee who responded. He was helpful and thoughtful, made it clear what the problem was, offered advice and promised to make whatever's possible on their end...

...only to have someone come up and act like a paying customer whose expectations weren't being met. He answered suggestions by saying something that comes down to "I don't understand, can you repeat please", and never quite grasped that if he wants a better experience for his users, he also needs to work for it.

The introduction to the response, in typical douche manager style, was the cherry on top.

Post reply on HN