I've been burned to many times by 3rd party HA solutions for Postgres. I'm not touching it again until they either have an official solution or if it is someone else's problem (eg. RDS).
Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
31–40 of 50 posts
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#32Zalando open source evangelist here. Some of my colleagues are on this thread, taking your questions. Meanwhile, wanted to plug a few other Postgres/K8s projects we're working on: -- Spilo: HA PostgreSQL cluster using Docker ( https://github.com/zalando/spilo ) -- kube-ingress-aws-controller: Configures AWS Load Balancers according to Kubernetes Ingress resources ( https://github.com/zalando-incubator/kube-ingress-aw…
I am curious did you look at Govenor(postres template for HA with etcd) before writing Patroni?
https://github.com/compose/governor
If you did can you comment on how these might compare and why you decided to write your own? Thanks!
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#33Zalando open source evangelist here. Some of my colleagues are on this thread, taking your questions. Meanwhile, wanted to plug a few other Postgres/K8s projects we're working on: -- Spilo: HA PostgreSQL cluster using Docker ( https://github.com/zalando/spilo ) -- kube-ingress-aws-controller: Configures AWS Load Balancers according to Kubernetes Ingress resources ( https://github.com/zalando-incubator/kube-ingress-aw…
I am curious did you look at Govenor(postres template for HA with etcd) before writing Patroni? https://github.com/compose/governor If you did can you comment on how these might compare and why you decided to write your own? Thanks!
I guess you did not look into Patroni's README, as it states:
> Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features.
Governor's git repo seems to have no recent activity (last commit 11 months ago).
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#34I've been burned to many times by 3rd party HA solutions for Postgres. I'm not touching it again until they either have an official solution or if it is someone else's problem (eg. RDS).
We also use RDS in Zalando (and it works great!). With running Patroni on Kubernetes we get several benefits over RDS which in our opinion warrant the effort. Some examples include:
* Real superuser access to the cluster
* Installing custom PostgreSQL extensions not available on RDS, PgQ (Queues in PostgreSQL), PgDecode (logical decode to Kafka/Json), PgJobs (minimalistic jobs in PostgreSQL)
* Easier migrations via S3/Streaming replication from and to AWS - RDS currently offers no way out of RDS to another PostgreSQL cluster
* Deployment model allows for more availability/flexibility with less costs - EBS + S3 allow us to run single node PostgreSQL with high uptime
* Multi node PostgreSQL can run on different node types
* OAuth login for PostgreSQL (via PAM)
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#35Earlier quoted context omitted.
I am curious did you look at Govenor(postres template for HA with etcd) before writing Patroni? https://github.com/compose/governor If you did can you comment on how these might compare and why you decided to write your own? Thanks!
I guess you did not look into Patroni's README, as it states: > Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features. Governor's git repo seems to have no recent activity (last commit 11 months ago).
Indeed I missed that, my bad. But anyway the README states:
>"Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features."
But it doesn't say what those features are. So I guess I'm curious what those features are without having to watch a whole youtube video. Or why they didn't submit PRs to Govenor for the added functionality.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#36Earlier quoted context omitted.
I guess you did not look into Patroni's README, as it states: > Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features. Governor's git repo seems to have no recent activity (last commit 11 months ago).
Indeed I missed that, my bad. But anyway the README states: >"Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features." But it doesn't say what those features are. So I guess I'm curious what those features are without having to watch a whole youtube video. Or why they didn't submit PRs to Govenor for the added functionality.
Compose guys are kind of happy with existing functionality of Governor and didn't really wanted to add something new. That was the main reason of making a fork.
If you read a further comments to this news you can get familiar with the part of new functionality of Patroni comparing to Governor.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#37Could any contributers to Patroni opine on what impact Postgresql 10's logical replication will have? Are you excited about it?
While logical replication is a long-awaited and exciting feature, I'd use it to replicate only a subset of my database, not as a replacement for HA. At present, it cannot follow the master in a physical replication after promotion (basically, you have to initialize your logical replicas from scratch), and it has a bigger lag then the physical replication, especially for long-running transactions, as one has to wait until the commit before sending the data.
Nevertheless, I think Patroni can help configuring logical replicas that will never be part of the HA (if one can tolerate reinitializing logical replicas after failing over). Pool requests are welcome!
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#38What's zalando/patroni's approach to performance scaling/load balancing? For example, pgpool2 can distribute SELECTs among multiple slaves which makes it useful for heavy analytics loads. Can you guys do that, or you only handle HA?
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#39Earlier quoted context omitted.
Indeed I missed that, my bad. But anyway the README states: >"Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features." But it doesn't say what those features are. So I guess I'm curious what those features are without having to watch a whole youtube video. Or why they didn't submit PRs to Govenor for the added functionality.
Compose guys are kind of happy with existing functionality of Governor and didn't really wanted to add something new. That was the main reason of making a fork. If you read a further comments to this news you can get familiar with the part of new functionality of Patroni comparing to Governor.
Thanks. Yeah I've been making my way through the video -
"Elephants on Automatic: HA Clustered PostgreSQL with Helm"
and its discussed at the 15:57 minute mark here if anyone else is interested:
https://www.youtube.com/watch?v=CftcVhFMGSY
Cheers.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#40What's zalando/patroni's approach to performance scaling/load balancing? For example, pgpool2 can distribute SELECTs among multiple slaves which makes it useful for heavy analytics loads. Can you guys do that, or you only handle HA?
Patroni has REST API providing a health-checks for load-balancers. patroni:8008/replica will respond with http status code 200 only if node is running as replica.
In the HAProxy config file you just need to list all Patroni nodes, specify a health-check and it will do a load-balancing for you.
The same approach works for example with AWS ELB.