Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bidirectional Replication is coming to PostgreSQL 9.6

blog.2ndquadrant.com

Post reply on HN
Print view View original

51–60 of 71 posts

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#51

Post by shabda »

dcosson wrote: post #19
anthony_franco wrote: post #9

Looking forward to playing around with this. Native master-master replication is the only thing keeping me on MySQL.

Just curious, what Postgres features are you missing on MySQL? I had only used MySQL until a year or two ago, and wondered what I was missing since Postgres seems to get more love/hype from the developer community for whatever reason. Now using Postgres in production, there are few if any features that I notice our team using which don't exist in MySQL (maybe Json landed in Postgres first is one big one?). One thing…

JSONFields for me. I want to use Aurora, but lack of JSONField makes it non-starter.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#52

Post by simon2Q »

ukj wrote: post #28

Holy crap, I am scared! Please, please, please read the fine print and ensure you understand the design tradeoffs as well as your application's requirements before blindly using this. The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? That should be interesting! No, none of that either - thi…

"Everything after "Basically" is misleading and inaccurate.

The conclusions here are not logical: The existence of some restrictions does not imply "Transactions are a lie", since that makes us think ALL transaction semantics are suspended, which is very far from the case.

There are good reasons for the design and restrictions in the BDR design, offering performance about x100 what is possible with eager consensus. Real world pragmatism, with production tools to ensure no inconsistencies exist in the database. The BDR tools allow you to trap and handle run-time issues, so it is not a theoretical debate or a mysterious issue, just a practical task for application authors to check their apps work.

"Consistent reads are a lie". Reads from multiple nodes at the same time are not guaranteed to be consistent - but this is multi-master - why would you read two nodes when all the data is on one node? The whole point is to put the data you need near the users who need it, so this is designed to avoid multiple node reads.

I could go on, and will do in a longer post elsewhere, but the main purpose of my retort is to show that the conclusions drawn here are not valid. Let's see how the hacker news method of consensus decides what is correct in this case.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#53

Post by simon2Q »

ioltas wrote: post #33
ukj wrote: post #28

Holy crap, I am scared! Please, please, please read the fine print and ensure you understand the design tradeoffs as well as your application's requirements before blindly using this. The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? That should be interesting! No, none of that either - thi…

It is amazing to see the number of projects based on Postgres mentioning that they are trying to achieve multi-master with their own restrictions: * Postgres-R is not multi-master, got discontinued in 2010, and lots most of its meaning after WAL-shipping has been added in-core with hot-standby. http://www.postgres-r.org/ * Postgres-XC, that says to do multi-master, the first project based on PostgreSQL that I heard o…

Postgres-R is dead. No need to look.

Postgres-XC has never implemented multi-master, so don't look there. Postgres-XZ, if it exists, is a closed-source fork of XC by a Chinese company.

Postgres-XL implements cluster scaling for queries and transactions. It is alive and well. http://blog.2ndquadrant.com/when-to-use-postgres-bdr-or-post...

X2 is the developers of XL and XC working together to see if we can help each other; there is no separate code from that initiative.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#55

Post by oliwarner »

pgaddict wrote: post #15
oliwarner wrote: post #13

As the developer who also manages the servers we deploy on, and not a full time PgDBA, things like multi-master replication scare the hell out of me. They really make me worry about what happens after downtime. And latency. Could anyone here recommend good reading material for scaling out your first database on to multiple servers? How do I know which scheme is the best for me?

The answer really depends on what you mean by "multi-master" - particularly whether you're looking for synchronous or asynchronous solution, what consistency model you need (strongly consistent cluster or nodes consistent independently), and what are your goals (write scalability, read scalability, disaster recovery, ...). BDR is meant to be asynchronous multi-master, i.e. a collection of nodes that are strongly cons…

We're dealing with a booking system so sync is important. But so is redundancy and throughout.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#56

Post by lmm »

gaius wrote: post #42
ukj wrote: post #28

Holy crap, I am scared! Please, please, please read the fine print and ensure you understand the design tradeoffs as well as your application's requirements before blindly using this. The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? That should be interesting! No, none of that either - thi…

The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? What's your use case here? Do you really want multi-master/bi-directional replication, or do you want distributed transactions? Why would an app want to connect to both masters, this isn't sharding?

If it isn't sharding then what is it? If every node has to process every write anyway (so there's no performance advantage over single-master) then why would one ever use this rather than traditional master-slave?

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#57

Post by lmm »

jeffdavis wrote: post #31
ukj wrote: post #28

Holy crap, I am scared! Please, please, please read the fine print and ensure you understand the design tradeoffs as well as your application's requirements before blindly using this. The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? That should be interesting! No, none of that either - thi…

Do you know of any relational products which offer high-throughput, low-latency, high-availability transaction processing using perfectly synchronous multi-master replication?

No. But I am aware of products that don't silently drop written data. That should be a pretty low bar.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#58

Post by gaius »

lmm wrote: post #56
gaius wrote: post #42

Earlier quoted context omitted.

The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? What's your use case here? Do you really want multi-master/bi-directional replication, or do you want distributed transactions? Why would an app want to connect to both masters, this isn't sharding?

If it isn't sharding then what is it? If every node has to process every write anyway (so there's no performance advantage over single-master) then why would one ever use this rather than traditional master-slave?

Active-active DR scenario, zero failover. Done this loads with Oracle. Or mostly-read regional DBs separated by a slow WAN, ditto.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#59

Post by lmm »

dcosson wrote: post #19
anthony_franco wrote: post #9

Looking forward to playing around with this. Native master-master replication is the only thing keeping me on MySQL.

Just curious, what Postgres features are you missing on MySQL? I had only used MySQL until a year or two ago, and wondered what I was missing since Postgres seems to get more love/hype from the developer community for whatever reason. Now using Postgres in production, there are few if any features that I notice our team using which don't exist in MySQL (maybe Json landed in Postgres first is one big one?). One thing…

MySQL has a bunch of irritating gotchas, most of which can be configured around or worked around, but they're just unpleasant by default. E.g. mysql "latin1" refers to a non-standard characterset that has only half its characters in common with the characterset commonly known as latin-1 (this is documented, but to find that documentation you have to first consider the posibility that "latin1" doesn't simply mean "latin1"). Likewise "utf8" refers to a non-standard characterset that has only 1/16th of its characters in common with the charcterset commonly known as UTF-8. A lot of its defaults are table-engine-specific, but the default table engine is set per-server, and even if you specify the engine you may silently get a table with a different engine (if the server doesn't have the specified engine enabled then it will use the default engine rather than giving an error). It uses non-standard quoting characters and non-standard table name equivalence by default. I can't remember what exactly went wrong with its timezone support but I remember it was bad enough that we didn't use it. A lot of its functions/columns will allow invalid values by default rather than failing to insert.

Re: Bidirectional Replication is coming to PostgreSQL 9.6

#60

Post by lmm »

gaius wrote: post #58
lmm wrote: post #56

Earlier quoted context omitted.

If it isn't sharding then what is it? If every node has to process every write anyway (so there's no performance advantage over single-master) then why would one ever use this rather than traditional master-slave?

Active-active DR scenario, zero failover. Done this loads with Oracle. Or mostly-read regional DBs separated by a slow WAN, ditto.

> Active-active DR scenario, zero failover.

I guess, but I don't see the advantage over master -> slave failover? You lose unreplicated writes either way.

> Or mostly-read regional DBs separated by a slow WAN, ditto.

If it's just reads then regional slaves work well. If your writes don't need to be transactional then a RDBMS seems unlikely to be a good fit in general.

Post reply on HN