Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CocoaPods downloads max out five GitHub server CPUs

github.com

Post reply on HN
Print view View original

131–140 of 320 posts

Re: CocoaPods downloads max out five GitHub server CPUs

#131

Post by masklinn »

D4AHNGM wrote: post #79
masklinn wrote: post #58

Earlier quoted context omitted.

OTOH Homebrew also uses github as its package host and has a respectable 62000 commits and 5600 contributors and github seems to be just fine with it. The big differences seems to be in the way they do their thing: I'm reasonably sure homebrew just git clones then updates the local repository normally[0], it has "only" 2500 files in Library/Formula, and because of its different subject it is way less write-active, Co…

Thankfully, none of Homebrew's binaries are stored on GitHub. Most of the changes committed to Homebrew are formulae-based; of the ~5600 contributors in Homebrew's lifetime only ~430 have contributed to the core code.

cocoapod's binaries are not stored in the problematic specs repository, cocoapods/specs only stores the equivalent of homebrew formulas (podspecs).

Re: CocoaPods downloads max out five GitHub server CPUs

#132

Post by miralabs »

noahlt wrote: post #115

Go's package manager, `go get`, also downloads from GitHub. I don't know the details of how `go get` and CocoaPods work, but I would be interested in learning why one is unscalable and the other seems to work.

probably on separate repos which something like npm and bower does as well.

Re: CocoaPods downloads max out five GitHub server CPUs

#134

Post by Gorbzel »

rmoriz wrote: post #81

CocoaPods (and Homebrew) mainly exist because of a lack of tooling in the typical Apple ecosystem. So I would blame Apple for not supporting the community with money or tooling. Letting GitHub with its limited amount of funding pay the bill isn't a nice move. Apple dev relations should throw some money at GitHub so they can provide some dedicated resources or offer to pay the cost of other solutions (like a 3rd party…

CocoaPods exists because developers want to learn how to "build apps" but lack the resources to intelligently include and link to 3rd party code in their projects. CocoaPods doesn't enable anything not otherwise configurable via git submodules and Xcode project hierarchies / build settings.

Therefore, it's not Apple's problem. In fact, I've talked to a non-trivial amount of engineers (both in Cupertino and long time Cocoa devs) that disapprove of the shortcuts that Cocoapods takes all over, software architecture be damned. Reasonable parties can agree to disagree, but I do include 3rd party framework inclusion without a dependency manager as an interview screen for prospective iOS hires.

Since you mention developer relations, I'll assume you're not actually arguing that this is Apple's technical responsibility, but that they should throw around some $ to grease the wheels to make dependency inclusion better. As a platform vendor, funding hosting costs for some project that you don't agree with just to "support the community" is a bad idea. Better idea is to allocate resources to setup a structure that can fix the issue in a technically agreeable way while also benefitting from the independence a FOSS project provides. In doing so, you are correct that it'd be preferable for Apple to fund/use well-known FOSS standards, such as Github.

In conclusion, Apple should setup a FOSS project to address the current inconveniences associated with third party package inclusion and should involve and pay Github somehow.

Oh wait...https://github.com/apple/swift-package-manager

Re: CocoaPods downloads max out five GitHub server CPUs

#135

Post by masklinn »

mitchtbaum wrote: post #103
pjc50 wrote: post #14

This reply: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm... "Not having to develop a system that somehow syncs required data at all means we get to spend more time on the work that matters more to us, in this case. (i.e. funding of dev hours)" In other words, using github as a free unlimited CDN lets them be as inefficient as they like. Such as having 16k entries in a directory ( https://github.com/Co…

Arch and Debian contributors have tried a good approach for package management.. 0: p2pacman - Bittorrent powered pacman wrapper 1: pacman & torrent, feasible? 2: DebTorrent (0) https://bbs.archlinux.org/viewtopic.php?id=163362 (1) https://bbs.archlinux.org/viewtopic.php?id=115731 (2) https://wiki.debian.org/DebTorrent I believe scaling this could happen with either: 1) lightweight filesystem\directory versioning sup…

> Arch and Debian contributors have tried a good approach for package management..

> 0: p2pacman - Bittorrent powered pacman wrapper

> 1: pacman & torrent, feasible?

> 2: DebTorrent

That's about distributing packages via p2p. The problematic repository doesn't store any package data, it stores package metadata (it's the cocoapods index if you will).

Re: CocoaPods downloads max out five GitHub server CPUs

#136

Post by Twirrim »

web007 wrote: post #46

I keep coming back to point #4 - who ever thought that 16k objects in a single directory would be a good idea? Ever since FAT that's been a bad idea, and while modern FSes will handle it without completely melting down it's still going to cause long access operations on anything to do with it. Even Finder or `ls` will have trouble with that, and anything with * is almost certainly going to fail. Is the use-case for t…

It's one of those shouldn't-be-arcane-but-somehow-is pieces of knowledge. Almost every job I've had I've ended up speaking to developers about more efficient file storage when I find yet another "shove everything in a single directory" implementation.

Re: CocoaPods downloads max out five GitHub server CPUs

#137

Post by sanjeetsuhag »

manyxcxi wrote: post #116
onli wrote: post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

> you're abusing

Cocoapods uses GitHub. No abuse here.

Re: CocoaPods downloads max out five GitHub server CPUs

#138

Post by mynameisvlad »

bobwaycott wrote: post #117
orclev wrote: post #45

Earlier quoted context omitted.

It seems particularly galling that their response to GitHub was to essentially throw their hands up and say "We don't want to change anything, fix it for us". I think GitHub had a near perfect response to this, they analyzed the problem, came up with a set of changes that could be made to help fix it (both short and long term), and pointed to steps they've taken to help out. CocoaPods on the other hand (or at least o…

I think that's pretty unfair. It's really obvious that the initial reply didn't really understand what was going on, and what was being explained. A couple followup additional explanations later, the same dev grokked the problem, CocoaPods' responsibility for the problem, and outlined a list of how they're going to solve it. Seemed to me to be a pretty nice example of professional and helpful candor between GH and an…

I don't know, maybe I'm being overly pessimistic here, but to me it just screams of backpedaling once they saw the reaction they were receiving in this thread. The position shifted from "it's the way we architected things, how can you fix this for us" to "okay, here's some things we can do" pretty quickly and dramatically when the HN thread went up and people were reacting to the response. Cocoapods is using Github resources for free, so the appropriate response from the start should have been what it eventually came down to, not pushing back on Github because they don't want to invest in an actual CDN solution. But, as I said, maybe I'm being overly pessimistic in my analysis here, that's just how it came off to me.

Re: CocoaPods downloads max out five GitHub server CPUs

#139

Post by tptacek »

manyxcxi wrote: post #116
onli wrote: post #2

Note how perfect that response from mhagger is. A clear, honest sounding assurance of what Github wants to deliver. A perfectly comprehensible description of what is the problem, and where it is coming from. And then suggestion how to fix it the project actually can work on, plus mentioning changes to git itself that Github is trying to make that would help. It not only shows great work going on behind the scenes (an…

I was astonished at how selfish/myopic/whatever alloy's response was. To be blunt, you're abusing the shit out of SOMEONE ELSE'S product that you're not even paying for. Your first question shouldn't be to see what Github can do for you to make it so you don't have to make changes. You should be falling over yourself investigating all available avenues for reducing load. It's an incredibly entitled way to think about…

I don't know, it sounded to me like he just didn't totally understand what Github was saying. By the end of the thread, it seemed like everyone was agreeing. I wouldn't be comfortable using words like "selfish" to describe any of what I read.

I certainly don't think the barb about your willingness to employ people who write things on Github issues threads that you disagree with is helping anyone understand any part of this situation. I understand the urge to find ways to be emphatic about how much you disagree with things, and I often find myself compelled to write lines like that, but I think they're virtually always a bad idea.

Re: CocoaPods downloads max out five GitHub server CPUs

#140

Post by masklinn »

pjc50 wrote: post #71
superuser2 wrote: post #31

Earlier quoted context omitted.

Perhaps because Cocoapods is not an OS package manager or anything close to it. It installs libraries within the context of an XCode project, regardless of the host system or what is installed for other projects.

It is a package manager, though? The fundamental idea of downloading a list of available options of which the user picks some, and the system pulls in dependencies, is almost exactly how dpkg and yum work. The location to which the packages are installed is a detail.

A language package manager must be able to "install" the same packages over and over again (and possibly "install" multiple versions of the same package in the same environment), and the ability to push packages is generally considered part of their duty, not so for OS package managers, you don't use dpkg to send a package to debian's repositories.

> The fundamental idea of downloading a list of available options of which the user picks some, and the system pulls in dependencies, is almost exactly how dpkg and yum work.

If you reduce it to the fundamentals you don't need yum or dpkg either to do that, just a dependency solver and curl.

Post reply on HN