Did you consider trying NewSQL DBs that would let you scale horizontally with much less complexity? Curious to see if the performance is comparable yet.
Are you talking about NoSQL databases, like Mongo and Redis?
Scaling the GitLab database
31–40 of 114 posts
Re: Scaling the GitLab database
#32for anyone new to gitlab, it is a bear. even their own cluster/HA documentation is mostly just a thin roadmap to a final goal. a statement of intent if you will. There are just far too many moving parts in this software and the glaring problem of "ruby doesnt scale well" is everpresent. packaged releases depend entirely on a bundled chef installation to get everything out the door and running.
The software tries to do too much. CI/wiki/code review/git/container repository/analytics package.
CI sharding per team and carving up gitlab into departments and groups has been my only solution so far, and its just distributed risk at that. its the difference between the unix ethos of do one thing and do it well, versus the F35 approach of do everything forever.
Re: Scaling the GitLab database
#33Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…
That's probably the best "feature comparison" page I've ever seen. (At least, the best written by a creator of one of the products being compared.)
Re: Scaling the GitLab database
#34for anyone new to gitlab, it is a bear. even their own cluster/HA documentation is mostly just a thin roadmap to a final goal. a statement of intent if you will. There are just far too many moving parts in this software and the glaring problem of "ruby doesnt scale well" is everpresent. packaged releases depend entirely on a bundled chef installation to get everything out the door and running. The software tries to d…
On the flipside, I quite enjoy Gitlab.
CI/CD that is very easy to use. I can self host it - so no $600 a month contract like some of the other guys. Decent Issues that are not as heavyweight as Jira. Place for code / CI/CD / Issues to all be in one place.
Re: Scaling the GitLab database
#35It's interesting to see what parts of the product GitLab is struggling with compared to GitHub, e.g. https://githubengineering.com/stretching-spokes/
Re: Scaling the GitLab database
#36Looking through the public Grafana dashboards I noticed that the database backups page is not showing any data points (http://monitor.gitlab.net/dashboard/db/backups). Is this meant to a public page?
Re: Scaling the GitLab database
#37Simple -- just clean the database from time to time [1].
BTW, I like how the banner on the bottom of [1] wants me to "Try GitLab Enterprise Edition risk-free for 30 days." currently.
[1] https://about.gitlab.com/2017/02/10/postmortem-of-database-o...
Re: Scaling the GitLab database
#38Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…
That's probably the best "feature comparison" page I've ever seen. (At least, the best written by a creator of one of the products being compared.)
[deleted]
Re: Scaling the GitLab database
#39for anyone new to gitlab, it is a bear. even their own cluster/HA documentation is mostly just a thin roadmap to a final goal. a statement of intent if you will. There are just far too many moving parts in this software and the glaring problem of "ruby doesnt scale well" is everpresent. packaged releases depend entirely on a bundled chef installation to get everything out the door and running. The software tries to d…
> and the glaring problem of "ruby doesnt scale well" is everpresent
Re: Scaling the GitLab database
#40Earlier quoted context omitted.
Not sure I fully follow on the multi node IO capacity, can you share a bit more on the workload that you're concerned about? Edit: Thanks for the clarification, the typo part in particular through me off, makes sense now.
Let's say that you're using RDS and have a single box capable of ~25k IOPS. If you have 16 boxes capable of some number of IOPS (let's say 15k), then your total system capacity is significantly higher than 25k. On read workloads where pages need pulled from disk (high read IOPS), this should see improvement in general. Secondaries cover this case for Gitlab, it seems, but that comes with a set of caveats as well (nam…
> Secondaries cover this case for Gitlab, it seems, but that comes with a set of caveats as well (namely async availability of data).
Another caveat is that all secondaries will end up having more or less the same stuff in their cache. As your data set grows bigger that becomes unsustainable, because you're going to read from disk more and more.
When you shard across N nodes you can keep N times as much data in the cache. Combined with N times higher I/O and compute capacity, that can actually give you way more than N times higher read throughput than a single node (for data sets that don't fit in memory), and you can get much higher write throughput as well.