Live data from Hacker News

Data Loss at GitLab

blog.2ndquadrant.com

101–110 of 236 posts

Re: Data Loss at GitLab

#101

With this incident, they once again showed that they are dedicated to transparency, even in the worst days. This increased their popularity on me and I believe among other developers. However, this may not be the case with the business people. I hope they can survive that and also publish a guide for getting better at the "ops" side of the things.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

> GitLab cost everyone a day

GitLab isn't popular because of the stability of its cloud platform. It's popular because you can install your own instance for free practically anywhere with minimal effort.

I run GitLab CE on a box in my server closet for projects that involve livelihoods.

Re: Data Loss at GitLab

#103
post #90

Earlier quoted context omitted.

SourceForge pretty much failed where GitHub strived though. For all practical purposes they were just a hosted CVS/SVN and nothing more. GitHub really nailed the idea of community and that made all the difference. Git is incidental, though lightweight branching was probably an important contributor to their approach.

I know the execution was a lot better with github, but what did it really add that source forge, google code, etc didn't do? They all had bug trackers, mailing lists, etc.

The killer feature is stupid easy forking and sending pull requests with the one interface.

Found some tiny project that does almost everything you want, but has a tiny feature you can implement easy? No worries, fork, add it.

Even if it's never pushed upstream, it's easy for someone to find and perhaps use themselves.

I do this with things like zabbix templates/scripts and docker containers that are 99% what I want.

Previously there was no easy way to do this - I'd have to set up an entirely new sourceforge/google code/etc account, which is a lot more effort.

Re: Data Loss at GitLab

#104
Honestly, I'm completely flabbergasted by this. Five backups, and NONE worked properly? Who made this? The S3 bucket was EMPTY? Has no one ever tested any of these backups?

It's not just the impact, which is fairly sizeable in it's own right, but it's the HUGE oversight on their part and the fact they tried to pin part of this on PostreSQL?

Credit where it's due; their report/transparency were good if a little unprofessional, and something I'd like to see more of from other companies.

Putting on my BOFH hat, this is what happens when you let Devs do operational stuff.

Re: Data Loss at GitLab

#105

Earlier quoted context omitted.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

There's another reasonable takeaway: GitLab will probably now prepare for this sort of thing better than the average competitor who hasn't been bitten by this.

Borges: "In Alexandria, it has been said that the only persons incapable of a sin are those who have already committed it and repented."

Re: Data Loss at GitLab

#106

With this incident, they once again showed that they are dedicated to transparency, even in the worst days. This increased their popularity on me and I believe among other developers. However, this may not be the case with the business people. I hope they can survive that and also publish a guide for getting better at the "ops" side of the things.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

The one who erred (and recovered) once is more valuable than the one who never made mistakes.

Re: Data Loss at GitLab

#107
post #90

Earlier quoted context omitted.

I know the execution was a lot better with github, but what did it really add that source forge, google code, etc didn't do? They all had bug trackers, mailing lists, etc.

The killer feature is stupid easy forking and sending pull requests with the one interface. Found some tiny project that does almost everything you want, but has a tiny feature you can implement easy? No worries, fork, add it. Even if it's never pushed upstream, it's easy for someone to find and perhaps use themselves. I do this with things like zabbix templates/scripts and docker containers that are 99% what I want.…

> The killer feature is stupid easy forking and sending pull requests with the one interface.

That's the part that wasn't really possible before git though. But you could always do an anonymous checkout and make your changes there,you could even do a pull request (via a patch). There was no need to signup for that, even though most of us had a source forge account (I still do) anyway.

Re: Data Loss at GitLab

#108

Earlier quoted context omitted.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

The one who erred (and recovered) once is more valuable than the one who never made mistakes.

There is no basis for that assertion.

Re: Data Loss at GitLab

#109

Earlier quoted context omitted.

HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

> GitLab cost everyone a day GitLab isn't popular because of the stability of its cloud platform. It's popular because you can install your own instance for free practically anywhere with minimal effort. I run GitLab CE on a box in my server closet for projects that involve livelihoods.

I know (because I use it and also host it at home, and like it). The point of my comment was HN's reaction.

Re: Data Loss at GitLab

#110

Earlier quoted context omitted.

Why? Everyone has their own copy of the gitlab remote.

We use gitlab.com for code reviews and CI, and we basically lost a day of integration productivity because of it. Yes, our process is too tightly tied to a single service, but it happens because we don't have the resources to self-host our own solution. We love GitLab, but this has absolutely got us looking elsewhere.

> We love GitLab, but this has absolutely got us looking elsewhere.

If you love GitLab and don't want to self host you can pay them for GitLab Hosted (paid customers had no troubles today).

Post reply on HN