Data Loss at GitLab

Post by ergo14 » Thu, Feb 02, 2017, 8:44 AM UTC

deckar01 wrote: post #101

praneshp wrote: post #93

Earlier quoted context omitted.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

> GitLab cost everyone a day GitLab isn't popular because of the stability of its cloud platform. It's popular because you can install your own instance for free practically anywhere with minimal effort. I run GitLab CE on a box in my server closet for projects that involve livelihoods.

Post by tscs37 » Thu, Feb 02, 2017, 9:19 AM UTC

E6300 wrote: post #121

thewhitetulip wrote: post #111

Earlier quoted context omitted.

>It's popular because you can install your own instance for free practically anywhere with minimal effort. If that is the reason why you use Gitlab, then why not try gitea or gogs? gogs is written in Go and provides a docker image or a drop in binary

Gogs is pretty sweet. After hearing about it here I installed it on my old home server with constrained memory because I couldn't meet Gitlab's (IMO) obscene memory requirements. It's perfect for my needs as a hobbyist and I even use it as a secondary Git server from work. It's also way, way faster than a self-hosted Gitlab or any public Git server, so even though my current server has more than enough memory to run…

  git config --global core.packedGitWindowSize 16m
  git config --global core.packedGitLimit 64m
  git config --global pack.windowMemory 64m
  git config --global pack.packSizeLimit 64m
  git config --global pack.thread 1
  git config --global pack.deltaCacheSize 1m

Post by mightbeconfused » Thu, Feb 02, 2017, 9:30 AM UTC

JohannesH wrote: post #143

iatanasov wrote: post #116

Earlier quoted context omitted.

I think you might be confused. One of the nice things to happen to OSS was the availability of kernel ( Linux kernel that is ) and git ( ... ) and both set very open models which allowed the OSS to trive. Github allowed for using git without knowing it .

I think you might be confused. One of the nice things to happen to OSS was the availability of transistors and electricity. Both made it possible to make millions of logic calculations per second without knowing it. :)

Post by sytse » Thu, Feb 02, 2017, 9:37 AM UTC

vollmond wrote: post #97

praneshp wrote: post #93

Earlier quoted context omitted.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

There's another reasonable takeaway: GitLab will probably now prepare for this sort of thing better than the average competitor who hasn't been bitten by this.

Post by sytse » Thu, Feb 02, 2017, 9:39 AM UTC

developer2 wrote: post #137

synicalx wrote: post #104

Honestly, I'm completely flabbergasted by this. Five backups, and NONE worked properly? Who made this? The S3 bucket was EMPTY? Has no one ever tested any of these backups? It's not just the impact, which is fairly sizeable in it's own right, but it's the HUGE oversight on their part and the fact they tried to pin part of this on PostreSQL? Credit where it's due; their report/transparency were good if a little unprof…

I agree. Their catastrophe is not the kind of thing that represents mild hiccups in operations that will be quickly resolved by the same employees who allowed this scenario to occur the first time around. This isn't the result of a single oversight. It screams of a systemic problem with the way the business operates - period. Maybe every project is rushed out with the deadline being the only metric that counts, quali…