Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Packages Is Down

githubstatus.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: GitHub Packages Is Down

#61

Post by crabbone »

deckard1 wrote: post #56
crabbone wrote: post #20

Earlier quoted context omitted.

Hello and welcome to PEP-508! In Python, we don't say "we don't host packages on a proprietary platform", we say "we have absolutely no clue where they are hosted and nobody audits them anyways, and we don't enforce package signing, and we'll just build from source with no build isolation what so ever, unless you remember to specify an obscure command-line option when installing... and have a nice day!"

it's amazing how we never learn. Things that Perl and CPAN and Linux distros figured out decades ago are constant issues today. It wasn't that long ago that NPM didn't even have checksums. CPAN runs unit tests on install. I can't imagine how slow that would be with npm. Package signing is, well... I suppose that's another lesson from the '90s people will learn about soon enough. With a web of trust as broad as python…

When it comes to my personal laptop, I also, typically, blindly trust the keys coming from developers because I don't have time for that. Not so much if I have to deploy a system into environment that several orders of magnitude more expensive than my laptop...

With systems like Python, I'd imagine that a solution to web of trust would be that some group of developers would organize a curated set of packages. So, for the cases where you need better security assurances, you'd use that. I mean, of course there's no guaranteed solution for the web of trust, but, in practical terms, something like that would be good enough for regulators.

There's already stuff like NumFOCUS. They don't particularly focus on the technical side of things, or endorsing more secure practices, but, in principle, they could. Maybe there will also be others once we have been bitten more times by some security breaches.

Re: GitHub Packages Is Down

#62

Post by JamesonNetworks »

naikrovek wrote: post #45
JamesonNetworks wrote: post #21

Earlier quoted context omitted.

For now, hopefully

yeah. if GitHub is even real .

Well, we know GitHub is real. But we should also remember that they can change their API at anytime and basing a package manager on their priorities is not the best situation for the long term success of that package manager, unless they are owned by Microsoft

Re: GitHub Packages Is Down

#64

Post by taeric »

schainks wrote: post #53
taeric wrote: post #35

Earlier quoted context omitted.

Mostly agreed, but I'd hazard a guess that the scale of github is far larger than distribution mirrors of old.

There are many distribution mirrors that are financed by universities that are on the the Internet backbone in the US Heck, even in Asia I did not have trouble with finding a good mirror.

I suspect this is the kind of advice that works for anyone, but would fail for everyone. That is, for most, it is a valid cost/benefit tradeoff to use the central option. Specifically, not just for them, but for everyone. If everyone was following this advice, it would likely start hitting scale/cost problems that would make running the mirrors of dubious value.

Re: GitHub Packages Is Down

#65

Post by booi »

crest wrote: post #59
Karellen wrote: post #50

Earlier quoted context omitted.

If only there were some way to make git distributed! /s

You mean something like a git annex enabled branch tracking mirror locations of each release artifact like HTTP URLs, (webseeded) torrents, maybe even something content addressed like IPFS? sigh

uhhh wait can you explain what you mean?

Re: GitHub Packages Is Down

#66

Post by dijit »

solatic wrote: post #57
lopkeny12ko wrote: post #34

What I find funny and unexplainable is that this class of problem was solved decades ago with distribution mirrors. It's not really clear to me why, within the last decade or so, we collectively decided to centralize hosting on one specific cloud service whose downtime now affects builds across nearly every company. What's perhaps even more surprising to me is that, after a repeated track history of severe and freque…

It's not even close to the same thing. Universities hosting mirrors piggybacked off academic networks; not just the computer kind but the social kind, where professors would regularly meet professors from other institutions at academic conferences. It was in the collective interests of the universities to set up mirrors to solve the pre-eminent issue of slow WAN networks. Today most companies need private package reg…

> the idea that GitHub is hosting everything in a single datacenter is laughable on its face.

Personally I find the notion that GitHub is somehow magically superior to the rest of the entire internet a bit silly.

I’ve worked on distributed systems my entire career, I have yet to find a single one that is completely immune to a datacenter outage, there is always some single point of failure not considered- often it is even known, everyone has the “special” datacenter.

Its also true that “market forces” push for better cost optimisation, which can, in cases, lead to being not sufficiently sized to cope with an outage of a whole DC- made worse are people who think cloud will solve this; because every customer will be doing the same thing as you during a zonal outage.

Regardless of that; you are basically suggesting that github, as a centralised system, is better equipped to deal with the distribution of packages than a literal distribution of package repositories?

That’s odd to me, maybe not “laughable”, but certainly odd.

Re: GitHub Packages Is Down

#67

Post by Xylakant »

mrweasel wrote: post #31

I'm disappointed that this is an issue for some package management systems. 20 years ago I helped run a mirroring service, it's still running today. Distributions such as Debian have hundreds of mirrors. This is a solved problems, but we just decided to but everything in the hands of one for-profit company.

I can’t speak for homebrew, but cargo/crate.io is fundamentally different from debian. The rate of churn is significantly higher, packages are published constantly and people expect them to be available. You can’t really do that with a system of mirrors like debian does. You can do edge caching and crates does that, but you want some central authority of which package versions are available. And every cargo run queries that index.

It’s acceptable for debian mirrors lag a few minutes or hours behind. The same thing is much harder to accept when the rate of change is much higher. Different requirements, different tradeoffs.

Re: GitHub Packages Is Down

#68

Post by mrweasel »

Xylakant wrote: post #67
mrweasel wrote: post #31

I'm disappointed that this is an issue for some package management systems. 20 years ago I helped run a mirroring service, it's still running today. Distributions such as Debian have hundreds of mirrors. This is a solved problems, but we just decided to but everything in the hands of one for-profit company.

I can’t speak for homebrew, but cargo/crate.io is fundamentally different from debian. The rate of churn is significantly higher, packages are published constantly and people expect them to be available. You can’t really do that with a system of mirrors like debian does. You can do edge caching and crates does that, but you want some central authority of which package versions are available. And every cargo run queri…

I guess I don't really understand the need for something like Cargo to be up to date to the seconds, or even minutes. My assumption is that you build your code with certain package versions in mind and release that to testing. Unless it's a security update, if won't matter if your 24 hours behind.

Say that version 1.2.1 of a library is release right as you do your build, that won't go into production within that 24 hour window anyway. If it is a security fix, then, like Debian, you pull that from another repository, which is under tighter control.

Re: GitHub Packages Is Down

#69

Post by kibwen »

Xylakant wrote: post #67
mrweasel wrote: post #31

I'm disappointed that this is an issue for some package management systems. 20 years ago I helped run a mirroring service, it's still running today. Distributions such as Debian have hundreds of mirrors. This is a solved problems, but we just decided to but everything in the hands of one for-profit company.

I can’t speak for homebrew, but cargo/crate.io is fundamentally different from debian. The rate of churn is significantly higher, packages are published constantly and people expect them to be available. You can’t really do that with a system of mirrors like debian does. You can do edge caching and crates does that, but you want some central authority of which package versions are available. And every cargo run queri…

> cargo/crate.io is fundamentally different from debian

Given the OP, note that packages on crates.io don't (and can't) reference Github. Crates.io has its own storage, and the only way to upload a crate to crates.io is if 100% of its dependencies are also on crates.io.

Re: GitHub Packages Is Down

#70

Post by pxc »

ddtaylor wrote: post #55
naikrovek wrote: post #37

Earlier quoted context omitted.

just yesterday I was stuck for an hour because a debian package mirror went down. took a long time to talk the user through changing their sources.list so that another mirror was chosen, and the mirror chosen out of that pool was down also. finally I had to manually check for a good mirror and give them the URLs. the user's take was "why don't they use GitHub packages?" "still running today" doesn't mean 100.0% uptim…

This is not how this works anymore. The system that is behaving this way must be relatively old at this point since almost all modern Debian based distros use the "mirror://" URI syntax now that automatically falls back to another mirror if one fails.

I don't think a clean Debian stable install uses that today.

But even so, at least the mirrorlist.txt file that appears in the mirror:// URI must be available for it to work, right?

Post reply on HN