Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling the GitLab database

about.gitlab.com

Post reply on HN
Print view View original

91–100 of 114 posts

Re: Scaling the GitLab database

#91

Post by pubgcrawler »

sytse wrote: post #56
vincentdm wrote: post #7

Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…

The good news is that page loads are much better now. Our initial page load ping http://stats.pingdom.com/81vpf8jyr1h9/1902794 is better than GitHub.com http://stats.pingdom.com/81vpf8jyr1h9/1902795 We got work to do in the 99% and merge request page load but the overall situation has improve dramatically. We still got work to do in availability, so I changed the 'feature' to reflect this https://gitlab.com/gitlab-co…

Maybe I just can't find it, but what URLs are pingdom.com checking? Do both issues have the same content? Or does one have 50 comments and another has 1?

Re: Scaling the GitLab database

#92

Post by anonacct37 »

mschuster91 wrote: post #81
YorickPeterse wrote: post #78

Earlier quoted context omitted.

You can have perfectly valid input escaping without needing prepared statements. Many ORMs handle this without the need for PREPARE/EXECUTE.

> You can have perfectly valid input escaping without needing prepared statements. People have bypassed this far too often for my taste, and there is a reason why SQL (and other injections) are Top 1 of OWASP (1st place actually, both in RC1 and 2 of 2017, but also in the 2013 edition). Never, ever trust any form of escaping or it will bite you. Hard.

How much do you trust the postgres developers?

https://www.postgresql.org/docs/9.1/static/libpq-exec.html#L...

Re: Scaling the GitLab database

#93

Post by tyingq »

vincentdm wrote: post #7

Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…

>In marketing terms, having fast page load would be called a "qualifier".

"Table stakes" is a common term for this.

Re: Scaling the GitLab database

#94

Post by sytse »

pubgcrawler wrote: post #91
sytse wrote: post #56

Earlier quoted context omitted.

The good news is that page loads are much better now. Our initial page load ping http://stats.pingdom.com/81vpf8jyr1h9/1902794 is better than GitHub.com http://stats.pingdom.com/81vpf8jyr1h9/1902795 We got work to do in the 99% and merge request page load but the overall situation has improve dramatically. We still got work to do in availability, so I changed the 'feature' to reflect this https://gitlab.com/gitlab-co…

Maybe I just can't find it, but what URLs are pingdom.com checking? Do both issues have the same content? Or does one have 50 comments and another has 1?

These are checking https://github.com/gitlabhq/gitlabhq/issues/1 and https://gitlab.com/gitlab-org/gitlab-ce/issues/1

The one on GitLab is considerably longer.

Re: Scaling the GitLab database

#95

Post by mjibson »

mschuster91 wrote: post #75

> A side effect of using transaction pooling is that you cannot use prepared statements, as the PREPARE and EXECUTE commands may end up running in different connections; producing errors as a result. Uhhh... WHAT? I'll simply sit by and watch for SQL injection vulnerabilities resulting from this change. Either you have a very good SQL query writer engine with rock solid escaping or you will get pwned by this.

In Postgres, beyond the SQL PREPARE commands, there is also a protocol for executing prepared statements. Their client drivers almost certainly use this protocol, which would mean the normal placeholder safety applies. I think this is just a misunderstanding of their point, and not a full backtracking to string escaping.

Re: Scaling the GitLab database

#96

Post by sytse »

the_duke wrote: post #80
nisa wrote: post #69

Earlier quoted context omitted.

So just use cgit on your 128mb box and don't complain about gitlab? It's like installing Windows 10 on a machine with 1gb memory, starting Photoshop and complaining about swapping and that it's running slow. Gitlab runs fine out of the box with 4gb memory and when screwing with some provided knobs it's fine with 2gb or less memory. It's in the docs, it's in every FAQ, every Stackoverflow answer... so why the hate? 64…

I'm running a Gitlab instance for a smallish dev team of 10. I haven't gotten it to run reliably with any less than 8Gb, even though we don't have a whole lot of activity there. It takes surprisingly long to start up too, even on a relatively beefy host. I do like the product though, the all-in-one solution with code hosting, issues, code review (needs work though...) and CI is great. (Not using the deployment, orche…

I'm surprised to hear GitLab isn't running reliably with 8GB. Are you running the Omnibus installations?

The open issues for CE https://gitlab.com/gitlab-org/gitlab-ce/issues do include feature requests.

Re: Scaling the GitLab database

#97

Post by personalcompute »

flukus wrote: post #86
mschuster91 wrote: post #75

> A side effect of using transaction pooling is that you cannot use prepared statements, as the PREPARE and EXECUTE commands may end up running in different connections; producing errors as a result. Uhhh... WHAT? I'll simply sit by and watch for SQL injection vulnerabilities resulting from this change. Either you have a very good SQL query writer engine with rock solid escaping or you will get pwned by this.

What is transaction pooling anyway? A transaction exists for a specific action, it shouldn't be shared among users or multiple connections. It sounds like they've taken transactions and removed the isolation. I don't know what the data access is like, but I get the impression that it's like some "enterprise" patterns I've seen where data access is too "encapsulated", ie, each DAL opens it's own connection, pulls it's…

"transaction-level" pooling might be a more apt description. Instead of assigning each incoming connection to a dedication upstream connection for the entire duration of the incoming connection, it assigns the upstream connections on a per-transaction basis. When each transaction ends, the upstream connection is returned to the pool. A better description is at https://wiki.postgresql.org/wiki/PgBouncer.

Re: Scaling the GitLab database

#98

Post by djd20 »

craigkerstiens wrote: post #19
tschellenbach wrote: post #3

Running PG bouncer is a very basic optimization. You typically start out with PG bouncer in your stack if you have experience running Postgres. If you're new to running your own postgres databases you should also check out Wall-e: https://github.com/wal-e/wal-e And the awesome pg stat statements https://www.postgresql.org/docs/10/static/pgstatstatements.h...

If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2]. [1] https://github.com/wal-g/wal-g [2] https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-...

While wal-g looks great, having evaluated it a bit, I worry that nobody seems to be maintaining it based on issues/pr's I looked at.

Re: Scaling the GitLab database

#99

Post by djd20 »

djd20 wrote: post #98
craigkerstiens wrote: post #19

Earlier quoted context omitted.

If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2]. [1] https://github.com/wal-g/wal-g [2] https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-...

While wal-g looks great, having evaluated it a bit, I worry that nobody seems to be maintaining it based on issues/pr's I looked at.

Scratch that - I see fresh commits - will give it another spin!

Re: Scaling the GitLab database

#100

Post by deepsun »

> 2. Use a connection pooler ...

I had my jaw dropped on that point -- they didn't use connection pooling? WTF? I'm seeing connection pools on pretty much any project I've worked on, even under 1/req/day load, it's pretty much "must have" since 90s. And it's really just a library (like dbcp2) and a few lines of config, with almost no drawbacks.

I cannot believe engineers at GitLab level are so bad at, well, engineering. I'm losing faith in their produce, please show me that I'm wrong.

Post reply on HN