Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling the GitLab database

about.gitlab.com

Post reply on HN
Print view View original

51–60 of 114 posts

Re: Scaling the GitLab database

#51

Post by treve »

vincentdm wrote: post #7

Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…

I love that they do though... It's so honest, and it gives me the feeling that I can trust that that might be the biggest of their problems, and not something else they don't want to talk about it.

Re: Scaling the GitLab database

#52

Post by ericfrederich »

Sounds like a design flaw... adding a centralized database to augment a decentralized version control system.

Comments, issues, pull requests, wiki, etc... should all be first class Git objects. I should be able to create pull requests while offline and push them up when I'm back online.

Re: Scaling the GitLab database

#53

Post by YorickPeterse »

RandomBK wrote: post #6

This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.

Yes, you can typically push PostgreSQL very far while still using a fairly simple setup (e.g. no sharding). Unfortunately too many times people have this mindset that a slow application is the result of a slow/bad database (as in "it's an RDBMS and RDBMS' don't scale"), and not the result of it being misused (e.g. badly written queries, lack of proper indexes, that sort of thing). At GitLab it took quite a while to get everybody to see this.

Re: Scaling the GitLab database

#54

Post by justinclift »

Not seeing any mention of using something like Memcached (or anything similar) in front of PG for caching.

Is that because it's already been done and not in scope?

Asking because for (at least) web applications, using Memcached (or similar) can significantly reduce the load on the backend database. :)

Re: Scaling the GitLab database

#55

Post by sb8244 »

mslot wrote: post #40
sb8244 wrote: post #20

Earlier quoted context omitted.

Let's say that you're using RDS and have a single box capable of ~25k IOPS. If you have 16 boxes capable of some number of IOPS (let's say 15k), then your total system capacity is significantly higher than 25k. On read workloads where pages need pulled from disk (high read IOPS), this should see improvement in general. Secondaries cover this case for Gitlab, it seems, but that comes with a set of caveats as well (nam…

> Secondaries cover this case for Gitlab, it seems, but that comes with a set of caveats as well (namely async availability of data). Another caveat is that all secondaries will end up having more or less the same stuff in their cache. As your data set grows bigger that becomes unsustainable, because you're going to read from disk more and more. When you shard across N nodes you can keep N times as much data in the c…

Great points, thanks!

Re: Scaling the GitLab database

#56

Post by sytse »

vincentdm wrote: post #7

Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…

The good news is that page loads are much better now. Our initial page load ping http://stats.pingdom.com/81vpf8jyr1h9/1902794 is better than GitHub.com http://stats.pingdom.com/81vpf8jyr1h9/1902795

We got work to do in the 99% and merge request page load but the overall situation has improve dramatically.

We still got work to do in availability, so I changed the 'feature' to reflect this https://gitlab.com/gitlab-com/www-gitlab-com/commit/3b3bddf5...

Re: Scaling the GitLab database

#57

Post by cbcoutinho »

ericfrederich wrote: post #52

Sounds like a design flaw... adding a centralized database to augment a decentralized version control system. Comments, issues, pull requests, wiki, etc... should all be first class Git objects. I should be able to create pull requests while offline and push them up when I'm back online.

I would rather baseline git be light and barebones - all the extra stuff is nice to have from a provider, but being part of git itself would make it bloat out.

Re: Scaling the GitLab database

#58

Post by sytse »

teraflop wrote: post #33
vincentdm wrote: post #7

Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…

That's probably the best "feature comparison" page I've ever seen. (At least, the best written by a creator of one of the products being compared.)

Thanks! We try to keep it fair. I love that our marketing team is on board with listing our missing features very publicly https://about.gitlab.com/features/#missing

Re: Scaling the GitLab database

#59

Post by sytse »

justinclift wrote: post #54

Not seeing any mention of using something like Memcached (or anything similar) in front of PG for caching. Is that because it's already been done and not in scope? Asking because for (at least) web applications, using Memcached (or similar) can significantly reduce the load on the backend database. :)

We have a lot of caching in Redis, but that connects with the Rails app, not in front of PG. This way it can store the result (for example a page fragment) instead of just the db query that is part of it.

Re: Scaling the GitLab database

#60

Post by sytse »

bzillins wrote: post #36

Looking through the public Grafana dashboards I noticed that the database backups page is not showing any data points ( http://monitor.gitlab.net/dashboard/db/backups ). Is this meant to a public page?

This is meant to be a public page. It loads async. After 3 seconds I see the following https://www.dropbox.com/s/5mt5f77e60yogk1/Screenshot%202017-...
Post reply on HN