Only pull requests and issues are gone. And even if the repo's were gone, wasn't git supposed to be a distributed vcs anyway? ;)
Data Loss at GitLab
61–70 of 236 posts
Re: Data Loss at GitLab
#62The gitlab situation and Uber's article speak to the level of immaturity of PGSQL's native replication feature, and more importantly: how not widely google-able nor documented/adopted the replication strategies are.
I believe gitlab used slony, not the native replication. I'm not well versed in postgres, but that's what I gleaned from reading their event log.
The google doc says at the bottom that slony only was used for a migration once, otherwise they use the replication features built into postgres.
Re: Data Loss at GitLab
#63Testing backups or at least monitoring them for correctness is a huge deal and it's problem I myself fudged up on one occasion, right around the holiday season which was terrible.
I've since setup wal-e with daily base backups deleting things older than a week along with the nightly pg_dump's along with a hot stanby. Maybe thats overkill, but after having lost data once. Never again!
The nice part about doing wal archiving like barman or wal-e do is you can do more than just backup/restore. You can do it with some time target in mind as well.
Someone somehow do a massive update or delete or insert millions+ in garbage? No worries, stop, destroy, restore to a previous point in time, continue onwards.
A bug in postgres or the kernel or the filesystem or any other multi-million line codebase in that stack screw up? Most likely the WAL segments are good up to a point still.
Hot standby gets you potentially sub-minute failovers if you automate them, or short enough to be ok even with manual failovers. WAL archiving gets you another whole level of safety net that is hard to beat.
Re: Data Loss at GitLab
#64While this is disastrous, I still think Gitlab is the best thing happened to OSS. This could be taken as a rhetoric, but on a more actionable side, we must all learn from Gitlab's experience. Almost everybody experiences this issue, but very few come out clean.
I think you are confused. The best thing to happen to OSS is GitHub, not GitLab. GitLab is just a fast follower and likely wouldn't even exist without the former. I for one am happy to throw money GitHub's way for their role in so dramatically changing how we code.
I pay for GitHub and can't deny their positive role in OSS, but I also use GitLab and will happily continue (and love that they have an OSS edition). I don't think it's a binary choice. Personally I will often adopt multiple pieces of software with redundant features because nuance often dictates the right tool for the job, and I enjoy being prepared for that.
Re: Data Loss at GitLab
#65Earlier quoted context omitted.
Note that GitLab is a for-profit company as well. It's just that it aligns better with the open source values.
..and they have a commercial offering too. Unlike some other things I've seen, at least their commercial offering isn't crippled and mostly has things that specifically cater to enterprise customers (like Active Directory integration. The community edition still has basic LDAP).
> like Active Directory integration. The community edition still has basic LDAP
I manage our company's GitLab instance. It's connected to our massive AD and it works just fine - the only thing I really miss is LDAP group creation and assignment. GitLab uses only the name and email attributes from LDAP/AD either way, and I think if I have some spare time I'll just write a hourly cronjob that manages groups and assignments using the GitLab API.
Re: Data Loss at GitLab
#66While this is disastrous, I still think Gitlab is the best thing happened to OSS. This could be taken as a rhetoric, but on a more actionable side, we must all learn from Gitlab's experience. Almost everybody experiences this issue, but very few come out clean.
I think you are confused. The best thing to happen to OSS is GitHub, not GitLab. GitLab is just a fast follower and likely wouldn't even exist without the former. I for one am happy to throw money GitHub's way for their role in so dramatically changing how we code.
You code differently using Git vs SVN? That's an interesting concept.
It doesn't change the way you code it just changes the way we share code.
Re: Data Loss at GitLab
#67Earlier quoted context omitted.
Yeah it made me not consider using them for anything other than toy projects.
Why? Everyone has their own copy of the gitlab remote.
Not everyone has their own copy of the project's issues/pull requests/settings/webhooks/CI config/etc., which the incident affected (and 6 hours worth were lost). Git repositories themselves do have the happy side effect that at least someone on your team has a full latest clone on their machine, but that's only because the source code is what you're there to use every day. Even if GitLab had a great "checkout your issues/etc. as a git repo" story, not many users would have an up-to-date copy backed up 100% of the time.
Re: Data Loss at GitLab
#68While this is disastrous, I still think Gitlab is the best thing happened to OSS. This could be taken as a rhetoric, but on a more actionable side, we must all learn from Gitlab's experience. Almost everybody experiences this issue, but very few come out clean.
I think you are confused. The best thing to happen to OSS is GitHub, not GitLab. GitLab is just a fast follower and likely wouldn't even exist without the former. I for one am happy to throw money GitHub's way for their role in so dramatically changing how we code.
Github was just a follower with a new SCM. The original would be source forge.
Re: Data Loss at GitLab
#69Earlier quoted context omitted.
I think you are confused. The best thing to happen to OSS is GitHub, not GitLab. GitLab is just a fast follower and likely wouldn't even exist without the former. I for one am happy to throw money GitHub's way for their role in so dramatically changing how we code.
You code differently using Git vs SVN? That's an interesting concept. It doesn't change the way you code it just changes the way we share code.
It doesn't change how I code. It changes how we code.
Re: Data Loss at GitLab
#70With this incident, they once again showed that they are dedicated to transparency, even in the worst days. This increased their popularity on me and I believe among other developers. However, this may not be the case with the business people. I hope they can survive that and also publish a guide for getting better at the "ops" side of the things.
All press is good press.