101–110 of 114 posts
This article is also very useful in showing just how far you can push Postgres _without_ reaching for any of these optimizations. I've seen too many projects worry about these things very early on in their lifecycle, when in reality they are no where close to having enough traffic to cause a problem.
> 2. Use a connection pooler ... I had my jaw dropped on that point -- they didn't use connection pooling? WTF? I'm seeing connection pools on pretty much any project I've worked on, even under 1/req/day load, it's pretty much "must have" since 90s. And it's really just a library (like dbcp2) and a few lines of config, with almost no drawbacks. I cannot believe engineers at GitLab level are so bad at, well, engineeri…
for anyone new to gitlab, it is a bear. even their own cluster/HA documentation is mostly just a thin roadmap to a final goal. a statement of intent if you will. There are just far too many moving parts in this software and the glaring problem of "ruby doesnt scale well" is everpresent. packaged releases depend entirely on a bundled chef installation to get everything out the door and running. The software tries to d…
Both the "everything all in one" approach, and their insistence on "omnibus" packages that bundle fucking everything into a giant deb/rpm.
Its unfortunate to me that there isn't a simple (as in cgit/gitweb/hgweb) oss repo web viewer that handles multiple repo types.
I want issues, ci, project management separate.
Sounds like a design flaw... adding a centralized database to augment a decentralized version control system. Comments, issues, pull requests, wiki, etc... should all be first class Git objects. I should be able to create pull requests while offline and push them up when I'm back online.
Git has native "pull requests" it's just not web based, it's email based.
A good wiki is markdown docs in a git/hg repo and something like Gollum to serve web viewers.
So - in a way those things can all be handled as regular content in git/hg.
We're moving away from gitlab because the performance is too much to bear. We're hearing "aaaaaarg I really hate gitlab" a few times a day. The pipeline pages takes 7 seconds to load. It's really bad. We stopped using issues because just listing them was a chore. Pushing 1 file takes at least 30 seconds to 1 minute. Anyways, the work falls on me to replace it. We're going with Phabricator and Jenkins.
Is that self-hosted Gitlab or their hosted version? Would you mind sharing details on your setup? Just curious as we're currently building out our own setup.
- Self hosted Gitlab CE
- Gitlab runners in pre-emptibles nodes with autoscaling
Future (almost done, still in testing phase):
- Phabricator (for git, code review and issues)
- Jenkins ("lightweight" CI, more details at the bottom)
- Google Cloud Container Builder ("Heavyweight" CI)
Our main repository is in Phabricator, but it has mirroring to Google Cloud Repository (for backup and faster heavyweight builds)
On commits, we run the lightweight CI in Jenkins. What I mean by lightweight is that, there's only a few runners and what it does is check what has changed. We have a mono-repository with over 30 microservices, rebuilding all of them is a waste of time and resources, so we have scripts and easy to use manifests that defines what to do. We also have a lot of small CI checks that is just not worth it to run externally, like linting.
Here's what our homegrown manifests looks like
build_graphql:
when_changed:
- graphql
- node_libraries/player-graphdb
- node_libraries/player-auth
- node_libraries/player-models
script_build: graphql/cloudbuild.yaml
script_skip: graphql/cloudbuild-skip.yaml
Let me know what else you want to know.
> 2. Use a connection pooler ... I had my jaw dropped on that point -- they didn't use connection pooling? WTF? I'm seeing connection pools on pretty much any project I've worked on, even under 1/req/day load, it's pretty much "must have" since 90s. And it's really just a library (like dbcp2) and a few lines of config, with almost no drawbacks. I cannot believe engineers at GitLab level are so bad at, well, engineeri…
There is a way to use prepared statements, it's called pre_prepare [0]. You still need to tweak your code though to not execute PREPARE directly but rather put statements in special table, but you do that only once, and after that you EXECUTE like before. Another minor (in my view) caveat is that you need to reset pgbouncer↔postgres connections each time you update that statements table.
Funny thing is that they publicly own up to their performance problems in an unusual way: in their comparison with GitHub ( https://about.gitlab.com/comparison/ ) they list "Fast page load" as a feature that GitLab lacks and GitHub has. Nevertheless, the slowness is really annoying, especially because their product is so good on all other accounts. If scaling their database can help speed things up, I bet they will b…
Running PG bouncer is a very basic optimization. You typically start out with PG bouncer in your stack if you have experience running Postgres. If you're new to running your own postgres databases you should also check out Wall-e: https://github.com/wal-e/wal-e And the awesome pg stat statements https://www.postgresql.org/docs/10/static/pgstatstatements.h...
If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2]. [1] https://github.com/wal-g/wal-g [2] https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-...
My team is currently using the hosted cituscloud, which uses PG Bouncer. They note that the reason for sharding is because of high writes. We've actually seen big benefits for moving over to a sharded setup via Citus just as much for the read performance. By sharding by customer we're able to more effectively leverage Postgres caching and elastically scale. Since switching over, our database has performed and scaled…
Craig from Citus here. Thanks for the kind words Justin, as you mention there is some upfront work, but once it's in place it becomes fairly manageable. We've been working to make that upfront work easier as well with libraries that allow things to be more drop-in (ActiveRecord-multi-tenant: https://github.com/citusdata/activerecord-multi-tenant and Django-multitenant: https://github.com/citusdata/django-multitenant…