Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Outage

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 87 posts

Re: GitHub Outage

#51

Post by LtWorf »

naikrovek wrote: post #38
RcouF1uZ4gsC wrote: post #31

Earlier quoted context omitted.

What else are you supposed to do when you can’t work because you don’t have access to your source code.

you still have access to your source code, you just can't push. or pull, but you can sneakernet around that, or have a second remote set in the repo for just this occasion, so you can collaborate as a stop-gap measure while GH gets fixed up.

git works completely fine offline.

However I have a feeling that most companies are set up to download 50MiB of dependencies at every run, so a website being down makes the entire thing not work.

Re: GitHub Outage

#53

Post by lysergia »

maxcan wrote: post #15

Github outages are the bored engineer's equivalent of getting a surprise snow day when you were in school, full of unbridled joy. For engaged, happy engineers its the equivalent of getting a surprise snow day when you are grown up and have to go dig your car out of the snow and its a normal day just with extra steps.

> equivalent of getting a surprise snow day

Not if you self-host Git

Re: GitHub Outage

#54

Post by view »

I'm trying to clone a repo at a whopping 6KB/s from Kenya.

EDIT: Seems to be a routing issue. I've enabled a UK VPN and it's working fine now.

Re: GitHub Outage

#55

Post by tpxl »

naikrovek wrote: post #29
adamscybot wrote: post #11

Earlier quoted context omitted.

In my experience its often the case the status site does not reflect reality -- until someone intervenes.

that's the whole point of a manually updated status page. you don't want automation to update it because that automation can fail. automation likely caused the outage you want to know more about. you also don't want your automation guessing at what the problem is, or what the effects are. you want real info from a real person even if it isn't given to you the millisecond you look for it. this is why status pages aren…

I'm going to disagree here. The point of a manually updated status page is appearances.

With proper reporting it's trivial to know which subsystem is experiencing problems, if any. It doesn't have to be very granular, just "normal", "experiencing issues", "offline". If reporting doesn't work, you should be alerted it doesn't work, and if alerting doesn't work, there needs to either be out-of-band alerting for that or someone monitoring the status at all times.

Manual overrides for status pages should exist for when the automation doesn't work of course.

At my last job we had a big screen in the office we monitored (Grafana) and we usually saw problems before the alerting kicked in - it had about a minute delay. When not in-office/during work hours, the on-call received alerts. It wasn't technically nor organisationally complex.

Re: GitHub Outage

#56

Post by pbhjpbhj »

MattIPv4 wrote: post #30
adamscybot wrote: post #11

Earlier quoted context omitted.

In my experience its often the case the status site does not reflect reality -- until someone intervenes.

Exactly, status pages tend to be updated by the humans responding to the incident, they're not automatic (that'd be pretty useless, you already know it's down, you want to know when they know it's down). Coordinating what to put on the status page when an incident happens can take time, getting the correct scope of impact from responding engineers etc.

Sorry, I'm not following you, how do you know it's down when the status page says it's all working? At that point you assume it's not down and start checking your own systems. They're just lying to avoid fallout; it's not better than an automated page.

Re: GitHub Outage

#57

Post by mejutoco »

MattIPv4 wrote: post #33
bowsamic wrote: post #14

Earlier quoted context omitted.

Status pages are absolutely useless. I've never seen them accurately reflect an outage

You are missing the point of a status page. They're not automatic things that tell you instantly when something is down -- that'd be pretty pointless, you already know it's down. They're updated by the folks responding to the incident, so you know they know there's an issue and that they're looking into it.

I agree with GP. If I am trying to, let's say, watch something on Netflix and it is not working, a status page would confirm that Netflix is down in my region, and I would know that there is nothing wrong with my connection, DNS, or any other potential cause.

Because of this reason I believe they would not be pointless if they were simply status pages, instead of "incident response pages". My hypothesis for them being this way instead is it is too much transparency for some companies for PR and legal reasons.

Re: GitHub Outage

#58

Post by b112 »

busymom0 wrote: post #43
HJain13 wrote: post #19

Earlier quoted context omitted.

Its Tuesday my man :)

They are probably Canadian (Canada had Monday off due to Labour Day, so people get confused).

Canadians are never confused, how dare you! I ... oh wait.

Sorry, was confused.

Re: GitHub Outage

#59

Post by hericium »

adamscybot wrote: post #11
omn1 wrote: post #2

Even though the status page ( https://www.githubstatus.com/ ) shows no issues, I'm still getting the occasional 500. It seems to be happening quite irregularly. They are possibly facing a lot of load.

In my experience its often the case the status site does not reflect reality -- until someone intervenes.

Static sites need editors and editors sometimes have to ask permission to post.

At least that's what AWS Health[1] looks like to me.

[1] https://health.aws.amazon.com

Re: GitHub Outage

#60

Post by capableweb »

lysergia wrote: post #53
maxcan wrote: post #15

Github outages are the bored engineer's equivalent of getting a surprise snow day when you were in school, full of unbridled joy. For engaged, happy engineers its the equivalent of getting a surprise snow day when you are grown up and have to go dig your car out of the snow and its a normal day just with extra steps.

> equivalent of getting a surprise snow day Not if you self-host Git

Self-hosting Git is easy, throw up a ssh server and point git to it.

Self-hosting everything else GitHub does is harder. Which is why they are building out all of those things, they don't want people to move to other places so easily.

Hopefully these constant outages makes more developers pissed off that issues are not stored in git as well, and start working on tooling to solve this shitty problem once and for all.

P2P/Local First software for everyone! \o/

Post reply on HN