171–180 of 236 posts
Earlier quoted context omitted.
How many days of productivity are you going to lose migrating off GitLab ?
And how do you ensure you're not going to have the same problem at the next place.
It feels like PostgreSQL is at the center of every terrible story about data loss[1] or poor performance[2]. I think companies prefer other databases like MySQL because they "just work." [1]: https://about.gitlab.com/2017/02/01/gitlab-dot-com-database-... [2]: https://eng.uber.com/mysql-migration/
The data loss in this incident was human error, not PostgreSQL fault.
Earlier quoted context omitted.
For those wondering about the live Gitlab stream, you can watch them work here: https://www.youtube.com/watch?v=nc0hPGerSd4
That is remarkably transparent. Commendable.
Earlier quoted context omitted.
I totally agree, the live stream [1] is amazing, discussing steps in the open like that and if there is some time left answer questions from the YouTube chat. [1] https://www.youtube.com/c/Gitlab/live
Glad to hear this was appreciated. It was an experiment -one we only hope to repeat in other scenarios.
Have a single point of contact that provides information about the recovery process. Being transparent and providing technical info is good, but that task should not be handled directly by the admins at the same time they are focusing on the drop-everything-shit-is-broke emergency.
Earlier quoted context omitted.
HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more. I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.
>GitLab cost everyone a day That's just nonsense. In most cases, the local repo should be more than enough to continue work.
Earlier quoted context omitted.
Github was just a follower with a new SCM. The original would be source forge.
SourceForge pretty much failed where GitHub strived though. For all practical purposes they were just a hosted CVS/SVN and nothing more. GitHub really nailed the idea of community and that made all the difference. Git is incidental, though lightweight branching was probably an important contributor to their approach.
Sourceforge had integrated mailing-list support, bug trackers, page-hosting, and more.
There was a community there, it was just that each community was based around a particular project. There was little chance of a user of project A from interesting with project B. But I guess the same could be said of github.
Sourceforge failed in part because of feature-creep, and availability issues. But I think it would be unfair to say that it wasn't "social".
Earlier quoted context omitted.
That is remarkably transparent. Commendable.
It sounds like a terrible working environment for the sysadmins. When shit is broke, you focus on fixing it. Being on camera is distracting, and setting up the livestream probably takes a bit of time that could be spent on actually fixing the problem.
Someone not involved with helping fixing the problem set up the stream from their home, while we continued work as normal.
I think the overall spirit was that it was comfortable to do it like this. Note that no one was required to work like this and we'd happily stopped streaming if anyone would have any problems with it.
With this incident, they once again showed that they are dedicated to transparency, even in the worst days. This increased their popularity on me and I believe among other developers. However, this may not be the case with the business people. I hope they can survive that and also publish a guide for getting better at the "ops" side of the things.
Earlier quoted context omitted.
Well, even if their transparency in failure is admirable, I'm not completely surprised. Just running an apt-get install gitlab gives me around ~350 dependencies. In those dependencies, I see python, ruby, nodejs, redis, postgres. With a little Java, and a little Go, plus some admin scripts written in Perl, the picture would be mostly complete... I may be a little harsh, but when I see a piece of software with so much…
Err, isn't Git the reasoning for that perl dependency? Iirc, Git depends on Perl for --interactive..