It would be awesome if you could do the same test with Stolon!
PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
21–30 of 43 posts
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#22I'm currently looking for similar info but for MySQL/MariaDB for an IoT side project ... any suggestions?
Depends what sort of solution you want.
There's regular single primary/n replica replication built in. There's no built in automatic failover.
There's also Group replication built in. This can be either single primary/n replica with automatic election of a new primary during a failure, or it can be multi-primary.
Then there's Galera, which is similar to the multi-primary mode of Group replication.
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#23> If the PostgreSQL backend is cancelled while waiting to acknowledge replication (as a result of packet cancellation due to client timeout or backend failure) transaction changes become visible for other backends. Such changes are not yet replicated and may be lost in case of standby promotion. This sounds like the two generals problem, which has no solution. But I may be misunderstanding.
Wouldn't the simple fix be to delay backend (= one connection) closing until all pending replication it initiated is finished?
That still leaves actual crashes, which would need to use the shared memory to store the list of pending replications before the recovery of transactions is finished.
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#24Great that nobody can track, or easily contribute to, the underlying postgres bug, because postgres has no issue tracker. Keeps the number of reported bugs nice and low. The discussion of critical bugs that lose your data is left to HN and Twitter threads instead.
Wow, they do public issue tracking in an unusual Way, via a mailing list: https://www.postgresql.org/list/pgsql-bugs/
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#25Is anyone here using YugabyteDB for high-availability Postgres? It seems like a compelling option: * Much closer to Postgres compatibility than CockroachDB. * A more permissive license. * Built-in connection manager [1], which should simplify deployment. * Supports both high availability and geo-distribution, which is useful if scaling globally becomes necessary later. That said, I don't see it mentioned around here…
Yugabyte is Postgres compatible, not actually Postgres. It's also only compatible insofar that you can use only a subset of Postgres features, as they're only supporting the most basics things like select, views etc Triggers, notifys etc were out of scope the last time I checked (which has admittedly been a while)
You're right of course, it's not entirely compatible, but this might be interesting:
> We use vanilla Postgres as-is for the query layer and replace Postgres storage with YugabyteDB’s own distributed storage engine.
https://www.yugabyte.com/blog/yugabytedb-enhanced-postgres-c...
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#26Great that nobody can track, or easily contribute to, the underlying postgres bug, because postgres has no issue tracker. Keeps the number of reported bugs nice and low. The discussion of critical bugs that lose your data is left to HN and Twitter threads instead.
Wow, they do public issue tracking in an unusual Way, via a mailing list: https://www.postgresql.org/list/pgsql-bugs/
So, like Linux? https://docs.kernel.org/admin-guide/reporting-issues.html
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#27Is there any alternative to Jepsen that does not involve writing spaghetti Clojure code?
Writing Clojure without spaghetti isn't too hard, and definitely more practical than waiting for a Jepsen alternative to come along.
The Jepsen author gave a great talk on all the performance engineering work that has gone into it, Jepsen is near enough an entire DBMS in its own right https://www.youtube.com/watch?v=EUdhyAdYfpA
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#28Is anyone here using YugabyteDB for high-availability Postgres? It seems like a compelling option: * Much closer to Postgres compatibility than CockroachDB. * A more permissive license. * Built-in connection manager [1], which should simplify deployment. * Supports both high availability and geo-distribution, which is useful if scaling globally becomes necessary later. That said, I don't see it mentioned around here…
I was under the impression that Yugabyte requires signing a CLA to contribute which leads me to avoid it for fear of them relicensing the thing when the VC's start squeezing. Also: very unique and single vendor driven. Seems like too much of a risk longer term but that is just my take. EDIT: in response to your question I did run a PoC of it but it had issues where I wasn't able to create very large indexes without t…
> DDL timeouts
That’s normal for building indices on large tables, regardless of the RDBMS. Increase the timeout, and build them with the CONCURRENTLY option.
> Query speed
Without knowing your schema and query I can’t say with any certainty, but it shouldn’t be dramatically slower than single-node Postgres, assuming your table statistics are accurate (have you run ANALYZE ?), necessary indices are in place, and there aren’t some horrendously wrong parameters set.
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#29I'm currently looking for similar info but for MySQL/MariaDB for an IoT side project ... any suggestions?
I've been running some small galera clusters for close to a decade with very good results. It's multi-master, and mostly "just works", though once every year or three I have to go in and re-bootstrap the cluster from the node with the highest sequence number if all of the nodes have gone down for one reason or another. I have some ansible playbooks and scripts that help with it.
My use case is a fairly small database of email address mappings. I have a mail server that sends out emails on behalf of our users, using their outside e-mail address (gmail, yahoo, whathaveyou). In order to allow replies, but prevent being an open relay, I have a milter that creates a lookalike address, with a token in it (user@example.com -> user_at_example_com_x1uif8dm@mailserver.example.net).
I store those mappings in the mysql database, less than 10K of those. So a trivial database, very high read to write ratio.
More recently, maybe 3-4 years ago, I added another couple tables that store all the logs. I have a job that reads the postfix logfiles and writes out all the log information so we can show users the status of every of their outgoing e-mails. That upped the amount of traffic to the database by quite a lot, but it's still pretty simple: basically no transactions, just simple single insert statements, and IIRC one overview table that gets some updates, then a clean up job to delete them after 30-90 days.
Galera has been a real workhorse.
For a while I was going to go with cockroachdb, and I set up a POC. It was pretty robust with setting it up and clustering, I never ran into having to re-bootstrap the couster. But, at the time, postfix couldn't write directly to cockroachdb because cockroach could only do UTF-8, and the postfix pgsql code would only do latin-1 or something. This has changed since and I've thought about switching back to cockroach, but I hear there are some licensing issues I may have to be aware of.
Re: PostgreSQL High Availability Solutions – Part 1: Jepsen Test and Patroni
#30Anyone familiar with autobase.tech?