Gitlab was down • Hacker News Board

Re: Gitlab was down

#11

Post by the-dude » Thu, Nov 28, 2019, 11:36 AM UTC

pearjuice wrote: post #6

I don't want to hijack this thread but what was the end result of the decision of Gitlab not hiring _in_ certain countries anymore? A lot of media coverage when the incident happened but no idea what happened afterwards and can't easily find it.

IIRC, they are not hiring 'in' certain counties. This is different than 'from' certain countries.

AFAIK, nothing material happened after the announcement. There are lots of companies which do not hire in certain countries.

Re: Gitlab was down

#12

Post by RossM » Thu, Nov 28, 2019, 11:37 AM UTC

kburman wrote: post #4

> Website, API, Git (ssh and https), Pages, Registry, CI/CD, Background Processing, Support Services, packages.gitlab.com, customers.gitlab.com, version.gitlab.com, forum.gitlab.com How come all of them are down all at once.

At the bottom of the page they list availability of third party services used - Fastly has a warning symbol, and I imagine they put that CDN in front of everything.

Re: Gitlab was down

#13

Post by bArray » Thu, Nov 28, 2019, 11:37 AM UTC

pearjuice wrote: post #6

I don't want to hijack this thread but what was the end result of the decision of Gitlab not hiring _in_ certain countries anymore? A lot of media coverage when the incident happened but no idea what happened afterwards and can't easily find it.

[Speculation] Perhaps a DDoS or a disgruntled employee...

In all seriousness, hopefully these things are not related.

Re: Gitlab was down

#14

Post by beredon » Thu, Nov 28, 2019, 11:37 AM UTC

Man, I just setup a new project on Gitlab an hour ago!

Re: Gitlab was down

#15

Post by bArray » Thu, Nov 28, 2019, 11:39 AM UTC

RossM wrote: post #12

kburman wrote: post #4

> Website, API, Git (ssh and https), Pages, Registry, CI/CD, Background Processing, Support Services, packages.gitlab.com, customers.gitlab.com, version.gitlab.com, forum.gitlab.com How come all of them are down all at once.

At the bottom of the page they list availability of third party services used - Fastly has a warning symbol, and I imagine they put that CDN in front of everything.

> Fastly has a warning symbol, and I imagine they put that

> CDN in front of everything.

Check their status page, it's just a simple reroute since the 13th [1].

[1] https://status.fastly.com/

Re: Gitlab was down

#16

Post by k_ » Thu, Nov 28, 2019, 11:40 AM UTC

kburman wrote: post #4

> Website, API, Git (ssh and https), Pages, Registry, CI/CD, Background Processing, Support Services, packages.gitlab.com, customers.gitlab.com, version.gitlab.com, forum.gitlab.com How come all of them are down all at once.

Latest tweet by @gitlabstatus said "We've identified an issue with database connectivity", which could explain why so many services are impacted.

Re: Gitlab was down

#17

Post by YorickPeterse » Thu, Nov 28, 2019, 11:40 AM UTC

jensvdh wrote: post #9

sojmq wrote: post #5

This strikes me as odd, because if there's something gitlab engineers are known for, is their unmatched competence,

Like when they dropped their database and took a few days to recover?

It took about 24 hours to recover, not "a few days".

Re: Gitlab was down

#18

Post by PhilKunz » Thu, Nov 28, 2019, 11:41 AM UTC

It seems like GitLab should indeed focus on stability now. The feature set is great. But it is all not worth it if it keeps disrupting work on a constant basis by being unstable.

Re: Gitlab was down

#19

Post by yyx » Thu, Nov 28, 2019, 11:42 AM UTC

manojlds wrote: post #10

beaconstudios wrote: post #7

Earlier quoted context omitted.

presumably a single point of failure - my guess would be something at the network level.

It's always DNS!

This time it might be Consul

Re: Gitlab was down

#20

Post by bArray » Thu, Nov 28, 2019, 11:42 AM UTC

Uninen wrote: post #8

We use GitLab at work and I use it for personal projects as well, it has been very slow for several days now and they had downtime yesterday as well. I don't mind the general sluggishness of the system that much (as I love the platform in general) but when you can't get your work (nor hobbies) done because of tools breaking, it gets really annoying really fast.

To be fair, just be thankful you don't use something like Jazz RTC - we had down time at least once every two weeks (from an hour to a whole day at a time). It got so bad we ended up setting up a local network Raspberry Pi as a Git server and emailing patches to remote teams.