CocoaPods downloads max out five GitHub server CPUs • Hacker News Board

Re: CocoaPods downloads max out five GitHub server CPUs

#71

Post by pjc50 » Tue, Mar 08, 2016, 4:55 PM UTC

superuser2 wrote: post #31

pjc50 wrote: post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

Perhaps because Cocoapods is not an OS package manager or anything close to it. It installs libraries within the context of an XCode project, regardless of the host system or what is installed for other projects.

It is a package manager, though? The fundamental idea of downloading a list of available options of which the user picks some, and the system pulls in dependencies, is almost exactly how dpkg and yum work. The location to which the packages are installed is a detail.

Re: CocoaPods downloads max out five GitHub server CPUs

#72

Post by jdreaver » Tue, Mar 08, 2016, 4:57 PM UTC

spoiler wrote: post #50

I find it amusing how GitHub's contact[1] form has (probably a recent addition): > GitHub Support is unable to help with issues specific to CocoaPods/CocoaPods. --- [1]: https://github.com/contact

I think that contact page remembers the last repo you visited. I went to it in incognito mode and is wasn't there.

That's a pretty neat feature!

Re: CocoaPods downloads max out five GitHub server CPUs

#73

Post by justinclift » Tue, Mar 08, 2016, 5:01 PM UTC

pjc50 wrote: post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

As a data point, "dnf" is the successor to yum. Started using it recently with Fedora 23... and it's pretty decent.

(It may not have been earlier on, I really don't know. ;>)

Something nifty about the new dnf is several of the older yum commands (eg builddep, yum-downloader) are now integrated directly so don't need extra utils installed. Seems like refinement is still happening.

If only my fingers didn't keep typing "dns" instead of "dnf" all the time, it would be great. :D

Re: CocoaPods downloads max out five GitHub server CPUs

#74

Post by rqebmm » Tue, Mar 08, 2016, 5:02 PM UTC

Gratsby wrote: post #11

From CocoaPods.org: > CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. It has over ten thousand libraries and can help you scale your projects elegantly. The developer response: > [As CocoaPods developers] Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (user…

1) I don't think they mean "scale your projects elegantly" in a "distribute your project to millions of customers" sense, but rather in a "add lots of libraries and not have it be a hassle" sense.

2) It makes perfect sense to let GitHub handle the performance hit until issues arise. Premature optimization is the devil, right? But once there start to be issues, it's definitely unfair to turn around and say "well you offer the service for free, so you should fix it"

Re: CocoaPods downloads max out five GitHub server CPUs

#75

Post by kuschku » Tue, Mar 08, 2016, 5:08 PM UTC

s_kilk wrote: post #56

wpeterson wrote: post #39

Earlier quoted context omitted.

RubyGems.org has their own web server and web services for publishing library versions and to allow the clients to fetch libraries and query the universal registry.

Same goes for almost any language-specific package manager you could name: Rust/Cargo, Clojure/Clojars, Node/npm, Elixir/hex, Python/PyPi. No matter which way you slice it, what CocoaPods is doing is a bit daft, especially at their scale.

Or even the Java community, where you have MavenCentral, JCenter, etc.

Re: CocoaPods downloads max out five GitHub server CPUs

#76

Post by stevoski » Tue, Mar 08, 2016, 5:08 PM UTC

athenot wrote: post #16

zymhan wrote: post #10

I've always found Github's business model interesting. What if a massive open-source organization (e.g. Fedora, Apache) decided to use it for all of their development, integrating it with continuous builds and all the associated pulls. Of course this isn't likely to happen for a number of reasons, but there are large open source projects that could put a significant load on their infrastructure if they chose to use G…

It's the same business model that Jira used in the past, when the alternatives were Mantis, Trac and Bugzilla. They had[1] one of the better and well designed issue trackers, free for OSS projects. That turned into a great way to champion adoption within paying customers' organizations. [1] In my opinion, they have since lost that edge on the UI.

I remember seeing this new much improved UI for bug tracking on OSS, and thinking, "Thank goodness they're no longer using Bugzilla". And then using it a bit, and thinking, "I've got to get that bug tracker into the org I'm working for." Yep, it was Jira. It was a great strategy they used!

Re: CocoaPods downloads max out five GitHub server CPUs

#77

Post by hayleox » Tue, Mar 08, 2016, 5:08 PM UTC

justinclift wrote: post #73

pjc50 wrote: post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

As a data point, "dnf" is the successor to yum. Started using it recently with Fedora 23... and it's pretty decent. (It may not have been earlier on, I really don't know. ;>) Something nifty about the new dnf is several of the older yum commands (eg builddep, yum-downloader) are now integrated directly so don't need extra utils installed. Seems like refinement is still happening. If only my fingers didn't keep typing…

I just keep typing "yum", since it's an alias of dnf (albeit with an annoying nag message) and since I work on CentOS servers a lot and am automatically used to typing "yum".

Re: CocoaPods downloads max out five GitHub server CPUs

#78

Post by justinclift » Tue, Mar 08, 2016, 5:09 PM UTC

riscy wrote: post #52

> Scaling and operating this repo is actually quite simple for us as CocoaPods developers whom do not want to take on the burden of having to maintain a cloud service around the clock (users in all time zones) or, frankly, at all. The CocoaPods developers seem to be missing the entire point of git: it's a _distributed_ revision control system. Setup a post-recieve hook on Github to notify another server, that is setu…

Rackspace is also known to sponsor significant resources for larger projects whom ask nicely. GlusterFS is one I used to be involved with doing this, and there are definitely others.

If CocoaPods reach out to Rackspace and/or other hosting providers, there's a decent chance they'll be able to pull together a good solution. :)

The downside though, is they'll need to figure out some way to keep it monitored/maintained. :/

Re: CocoaPods downloads max out five GitHub server CPUs

#79

Post by D4AHNGM » Tue, Mar 08, 2016, 5:10 PM UTC

masklinn wrote: post #58

revelation wrote: post #35

Earlier quoted context omitted.

They are using GitHub for their intended purpose, hosting their code. That is perfectly fine. What is not perfectly fine is using GitHub as your package host, CocoaPods/Specs is the equivalent of Debians APT using one big GitHub repo to host all their packages. It has 92567 commits and 6872 contributors.

OTOH Homebrew also uses github as its package host and has a respectable 62000 commits and 5600 contributors and github seems to be just fine with it. The big differences seems to be in the way they do their thing: I'm reasonably sure homebrew just git clones then updates the local repository normally[0], it has "only" 2500 files in Library/Formula, and because of its different subject it is way less write-active, Co…

Thankfully, none of Homebrew's binaries are stored on GitHub.

Most of the changes committed to Homebrew are formulae-based; of the ~5600 contributors in Homebrew's lifetime only ~430 have contributed to the core code.

Re: CocoaPods downloads max out five GitHub server CPUs

#80

Post by jessaustin » Tue, Mar 08, 2016, 5:10 PM UTC

superuser2 wrote: post #18

Just last night, all my pod installs were timing out after ~30ish minutes. That explains it.

Should we infer from your behavior that such a timeframe is fairly normal for cocopod? If I ran an "npm i" (or "pip install", etc.) that didn't respond for a minute, I would suspect a problem and kill it. How can any development process that takes longer than that be practical?