Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
1–10 of 50 posts
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#2How does this compare to other HA postgres products like stolon or Postgres-XL?
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#3How does this compare to other HA postgres products like stolon or Postgres-XL?
As per my understanding, Postgres-XL is not comparable to Patroni. Postgres-XL is comparable to PostgreSQL. Postgres-XL shards the data across multiple data nodes. Where as Patroni uses etcd, consul or zookeeper to provide HA for any Postgres cluster using replication (Data is written only to a single instance and replicated further).
I think you meant Stolon. Stolon is quite comparable to Patroni and has more features than Patroni, but I am not sure how many people are using it. I found more resources for Patroni as compared to stolon.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#4How does this compare to other HA postgres products like stolon or Postgres-XL?
Patroni makes PostgreSQL cloud native; easily add and despose nodes in a self healing postgres cluster. Think of it as nosql style cluster management. PostgreSQL XL is more "classic" (non self healing) clustering.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#5This is very interesting. I thought it was best to keep yr db out of the k8s cluster. But this seems to make Postgres "cloud native".
The main reason for keeping dbs out of k8s was that the storage (persistence) solutions in k8s were not up to the task yet. Now I wonder how is Patroni doing persistence? I cannot find anything on it in the Patroni docs. Maybe Patroni is so "self healing" that a proper storage solution is not an issue -- dunno, just guessing here.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#6This is very interesting. I thought it was best to keep yr db out of the k8s cluster. But this seems to make Postgres "cloud native". The main reason for keeping dbs out of k8s was that the storage (persistence) solutions in k8s were not up to the task yet. Now I wonder how is Patroni doing persistence? I cannot find anything on it in the Patroni docs. Maybe Patroni is so "self healing" that a proper storage solution…
I am not really sure Patroni has anything to do with the persistence of data. It just uses etcd, zookeper or consul to elect a master in case of a failover and uses their key value store to save the information about the current master. Its upto you whether you keep your database on a container or not. And how the data is managed by the container.
This talk by Josh Berkus explains how Patroni works pretty well.
https://www.youtube.com/watch?v=OH9WSEiMsAw
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#7This is very interesting. I thought it was best to keep yr db out of the k8s cluster. But this seems to make Postgres "cloud native". The main reason for keeping dbs out of k8s was that the storage (persistence) solutions in k8s were not up to the task yet. Now I wonder how is Patroni doing persistence? I cannot find anything on it in the Patroni docs. Maybe Patroni is so "self healing" that a proper storage solution…
I am not really sure Patroni has anything to do with the persistence of data. It just uses etcd, zookeper or consul to elect a master in case of a failover and uses their key value store to save the information about the current master. Its upto you whether you keep your database on a container or not. And how the data is managed by the container. This talk by Josh Berkus explains how Patroni works pretty well. https…
Thanks. All the k8s talk in the README confused me I guess.
And that's a great talk, made Patroni much clearer for me.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#8How does this compare to other HA postgres products like stolon or Postgres-XL?
As per my understanding, Postgres-XL is not comparable to Patroni. Postgres-XL is comparable to PostgreSQL. Postgres-XL shards the data across multiple data nodes. Where as Patroni uses etcd, consul or zookeeper to provide HA for any Postgres cluster using replication (Data is written only to a single instance and replicated further). I think you meant Stolon. Stolon is quite comparable to Patroni and has more featur…
> Stolon is quite comparable to Patroni and has more features than Patron.
Let me as a question, what feature Stolon has and Patroni doesn't? Can you give an example?
From my side I can provide list of features available in Patroni, but not in Stolon. For example:
* there is no way to do a controlled failover (switchover) in Stolon.
* in Patroni it's possible to exclude some nodes from a leader race.
* Patroni supports cascading replication
* Patroni can take basebackup from replicas if they are marked with a special tag
* it's even possible to configure Patroni to use a custom backup/recovery solution instead of pg_basebackup
* Patroni can give you a hint that postgres must be restarted to apply some configuration changes
* with Patroni it's even possible to schedule switchover or restart restart of postges on some specific time (for example 04:00 am, when traffic is minimal)
* with Patroni you can control High-Availability / Durability ratio. I.e.: what to do if master postgres has crashed? Either you will try to start it back or failover to a replica. But start of a crashed postgres may take a long time. With Patroni you can configure that if postgres didn't started in some amount of seconds - please do failover.
Stolon provides cloud-native deployment and Patroni doesn't? This is not really true. The main idea of Patroni is to be not dependent on some specific technologies. It can work as on bare-metal with the same success as on Kubernetes. It's very easy to build a custom solution with Patroni for your use-case. For example there is a Spilo project: https://github.com/zalando/spilo, a docker image build with Patroni and wal-e for a cloud deployments.
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#9Could any contributers to Patroni opine on what impact Postgresql 10's logical replication will have? Are you excited about it?
Re: Patroni: A Template for PostgreSQL HA with ZooKeeper, Etcd, or Consul
#10This is very interesting. I thought it was best to keep yr db out of the k8s cluster. But this seems to make Postgres "cloud native". The main reason for keeping dbs out of k8s was that the storage (persistence) solutions in k8s were not up to the task yet. Now I wonder how is Patroni doing persistence? I cannot find anything on it in the Patroni docs. Maybe Patroni is so "self healing" that a proper storage solution…
There are many solutions for network block device style persistent storage in K8S now. In the cloud, you have solutions like Amazon's EBS and GCE's persistent disks. On VMware you have local VMDKs. On your own equipment, there's support for iSCSI, Ceph RBD, OpenStack Cinder, Portworx, ScaleIO, Quobyte, and just about anything else if you write a few scripts to format and mount volumes on the host (FlexVolume). Except for FlexVolume, K8S is able to dynamically provision volumes of a requested size for you.
If you're feeling adventurous there's support for network filesystems like NFS, GlusterFS, CephFS, but I wouldn't recommend those for Postgres.
https://kubernetes.io/docs/concepts/storage/persistent-volum...
In one of his demos, Josh Berkus recommends just using the K8S ephemeral "emptyDir" storage type on each node, and counting on Patroni to maintain data persistence through replication. e.g. any one Postgres node may die and lose its local data, but when its replacement is created, it will be populated with a fresh copy of the database wherever it is scheduled on the K8S cluster. That sounds workable for smaller databases, though it gives every single person I've suggested it to nightmares.