GitHub: October 21 Incident Report • Hacker News Board

Re: GitHub: October 21 Incident Report

#11

Post by skocznymroczny » Mon, Oct 22, 2018, 11:09 AM UTC

nik736 wrote: post #6

I don't understand why code hosting platforms like GitHub, GitLab or BitBucket have so many issues regularly. Is there anything special about it?

Lots of users. Also, a lot of package managers rely on hosting platforms like GitHub to host their packages, so if Github breaks, a lot of CI processes around the world break.

Re: GitHub: October 21 Incident Report

#12

Post by jhabdas » Mon, Oct 22, 2018, 11:13 AM UTC

madeofpalk wrote: post #9

nik736 wrote: post #6

I don't understand why code hosting platforms like GitHub, GitLab or BitBucket have so many issues regularly. Is there anything special about it?

It’s hard. But really, do they have that many problems?

This is normal during an acquisition

Re: GitHub: October 21 Incident Report

#13

Post by JAdamMoore » Mon, Oct 22, 2018, 11:15 AM UTC

It won't keep me logged in long enough to make any changes or do anything. I'd rather not have access than have time wasting, useless access.

Re: GitHub: October 21 Incident Report

#14

Post by laumars » Mon, Oct 22, 2018, 11:15 AM UTC

nik736 wrote: post #6

I don't understand why code hosting platforms like GitHub, GitLab or BitBucket have so many issues regularly. Is there anything special about it?

> Is there anything special about it?

Yup. You notice when they're down. Whereas when your self-hosted git server goes offline for a couple of hours, nobody else notices.

Re: GitHub: October 21 Incident Report

#15

Post by deathhand » Mon, Oct 22, 2018, 11:23 AM UTC

jbb67 wrote: post #3

Ugh I detest the term " abundance of caution " as used in that message. Weasel words designed to stop you thinking about the problem as who wouldn't want them to be overly cautious

It's not that bad. X definitely modifies Y but it may also modify Z. You are informing the audience of this fact and it provides them with 'warm fuzzies'.

Re: GitHub: October 21 Incident Report

#16

Post by josteink » Mon, Oct 22, 2018, 11:24 AM UTC

> Further, this incident only impacted website metadata stored in our MySQL databases, such as issues and pull requests.

Basically the only things I was going to do on Github today :)

Re: GitHub: October 21 Incident Report

#17

Post by TeMPOraL » Mon, Oct 22, 2018, 11:24 AM UTC

skocznymroczny wrote: post #11

nik736 wrote: post #6

I don't understand why code hosting platforms like GitHub, GitLab or BitBucket have so many issues regularly. Is there anything special about it?

Lots of users. Also, a lot of package managers rely on hosting platforms like GitHub to host their packages, so if Github breaks, a lot of CI processes around the world break.

Which is kind of ridiculous. If your CI breaks because GitHub is down, it means it's not caching dependencies locally, but keeps re-downloading them every time it runs (e.g. every commit), generating tons of waste and unnecessary load on the hosting service.

Or, to put it bluntly, if your CI works like this, it's contributing to climate change.

Re: GitHub: October 21 Incident Report

#18

Post by richardwhiuk » Mon, Oct 22, 2018, 11:27 AM UTC

I hope they are going to provide a better RCA than that!

Re: GitHub: October 21 Incident Report

#19

Post by realreality » Mon, Oct 22, 2018, 11:28 AM UTC

TeMPOraL wrote: post #17

skocznymroczny wrote: post #11

Earlier quoted context omitted.

Lots of users. Also, a lot of package managers rely on hosting platforms like GitHub to host their packages, so if Github breaks, a lot of CI processes around the world break.

Which is kind of ridiculous. If your CI breaks because GitHub is down, it means it's not caching dependencies locally, but keeps re-downloading them every time it runs (e.g. every commit), generating tons of waste and unnecessary load on the hosting service. Or, to put it bluntly, if your CI works like this, it's contributing to climate change.

That’s an excellent point. How can you tell if the CI system uses caching, other than waiting for a github outage to notice something broke?

Re: GitHub: October 21 Incident Report

#20

Post by TeMPOraL » Mon, Oct 22, 2018, 11:34 AM UTC

realreality wrote: post #19

TeMPOraL wrote: post #17

Earlier quoted context omitted.

Which is kind of ridiculous. If your CI breaks because GitHub is down, it means it's not caching dependencies locally, but keeps re-downloading them every time it runs (e.g. every commit), generating tons of waste and unnecessary load on the hosting service. Or, to put it bluntly, if your CI works like this, it's contributing to climate change.

That’s an excellent point. How can you tell if the CI system uses caching, other than waiting for a github outage to notice something broke?

Check in documentation, or when in doubt, run a job, redirect GitHub to 127.0.0.1 in /etc/hosts on the CI server, and run that job again.