Other options available: - CockroachDB 1.1 - AWS RDS Aurora PostgreSQL-compatibility [1] https://news.ycombinator.com/item?id=15458900 [2] https://news.ycombinator.com/item?id=13072861
Scaling the GitLab database
21–30 of 114 posts
Re: Scaling the GitLab database
#22Earlier quoted context omitted.
User mentioned "trying". With an app that gets this much use, it might be reasonable to take a small piece and fork all writes to both the current + an experimental DB and then A/B test the reads. If this type of abstraction requires "a complete rewrite of GitLab" for "trying NewSQL DBs" then something is amiss.
Given that Gitlab is open source, adding a NewSQL DB into the mix would be pushing that dependency to downstream users, who would then have to take on the maintenance burden. It may end up being the right call from a performance standpoint but just too complex to ask users to manage.
Re: Scaling the GitLab database
#23This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
Re: Scaling the GitLab database
#24Re: Scaling the GitLab database
#25This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
Is easy to overlook how much you can do with a decent RDBMS. And for the small data that gitlab use, I believe still exist a lot of big wins on performance.
Is just that the new generation not pay much attention to Sql databases...
P.D: I don't mean the gitlab developers, just on general
Re: Scaling the GitLab database
#26Other options available: - CockroachDB 1.1 - AWS RDS Aurora PostgreSQL-compatibility [1] https://news.ycombinator.com/item?id=15458900 [2] https://news.ycombinator.com/item?id=13072861
Aurora RDS-PG still isn't GA, as in isn't available in all datacenters. Edit: Apparently it is GA as of 6 days ago, but still only available in 4 regions https://aws.amazon.com/blogs/aws/now-available-amazon-aurora...
Re: Scaling the GitLab database
#27This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
Re: Scaling the GitLab database
#28Re: Scaling the GitLab database
#29This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
The whole point why you should worry from the beginning is so that you don't have to re-work everything and put a huge risk to the business when you have to do it.
Edit: I would also add that most projects I've seen tend to undergo one or more large refactors / redesigns before growing to a size where complex DB scaling is needed. This is, of course, speaking from the perspective of a small startup or pet project. If you're writing a new feature for an existing product where you can assume that it'll have lots of users, then you would obviously build scaling right from the start.
Re: Scaling the GitLab database
#30This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
The whole point why you should worry from the beginning is so that you don't have to re-work everything and put a huge risk to the business when you have to do it.