Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

pg_rewind in PostgreSQL 9.5

hlinnaka.iki.fi

Post reply on HN
Print view View original

41–50 of 63 posts

Re: pg_rewind in PostgreSQL 9.5

#41

Post by lobster_johnson »

moe wrote: post #33
rosser wrote: post #32

Earlier quoted context omitted.

No, and nor did I mention "fencing". I said "minimum viable". No other open source RDBMS manages those things, so calling postgres out for lacking them is a little disingenuous. Replication isn't "inherently" anything. Making it a runtime thing is one way of doing it . Just because it's your preferred way (or — as often seems to be the case among people who share your opinion about how postgres does it — just how the…

fencing Which relates to a master/slave cluster as in how? no other open source RDBMS out there manages those things either Rethinkdb, Redis, Riak, ElasticSearch all do. (them not being relational is irrelevant to replication) Making it a runtime thing is one way of doing it. It's the only way of doing it for modern, dynamic application layouts (the whole "cloud" thing). The stereotypical static database cluster, car…

> Rethinkdb, Redis, Riak, ElasticSearch all do.

None of those are RDBMSes. Since they all lack schemas, constraints, strict consistency and ACID transactions, their replication logic can be much simpler. There are some very good reasons that true multimaster replication is nearly unknown among relational databases.

Re: pg_rewind in PostgreSQL 9.5

#42

Post by lobster_johnson »

moe wrote: post #36
rosser wrote: post #34

Earlier quoted context omitted.

fencing ... relates to a master/slave cluster as in how? Because when you promote a slave, you also need to fence writes to the old master, otherwise you can end up in a "split-brain" scenario, where the old and new masters are both accepting writes, and nobody's replicating. Have fun cleaning that up. (them not being relational is irrelevant to replication) I disagree. Their being relational imposes extra constraint…

Because when you promote a slave, you also need to fence writes to the master, otherwise you can end up in a "split-brain" scenario. There is only one master, hence there can not be a "split brain". You seem to be confusing master/slave and multi-master configuration (which pg doesn't support). I disagree. Their being relational imposes extra constraints upon replication. ACID? Well, it doesn't. Look, I'm rapidly los…

> There is only one master, hence there can not be a "split brain".

That's very naive. There are situations -- very real ones -- where the failing master doesn't know it is failing, where clients can still connect and the new master can't safely tell the old one to stop being a master. This is where fencing comes in, and other concepts like heartbeats and STONITH and backup interconnects and so on, all of which makes automatic failover complicated and error-prone.

There are good reasons that 35+ years after the invention of the relational database, only the top commercial ones implement this really cleanly (but they still require expensive dedicated DBAs, for the most part).

Re: pg_rewind in PostgreSQL 9.5

#43

Post by [deleted] »

moe wrote: post #23
olefoo wrote: post #22

Earlier quoted context omitted.

I try to use 'primary' and 'replica'; but depending on who I'm talking to I find myself slipping back to the Master/Slave terminology. And to the OP, replication is a complex process fraught with security risks. Take about 10 minutes thinking about just roles and authentication in the context of your proposed syntax and what you would need to set up ahead of time to make it work. The current method of starting with a…

what you would need to set up ahead of time to make it work Nothing except a shared secret or keypair. Other databases (rethinkdb, redis, riak etc.) show how it's done.

[deleted]

Re: pg_rewind in PostgreSQL 9.5

#44

Post by waterhouse »

moe wrote: post #18
afarrell wrote: post #15

Earlier quoted context omitted.

> enslave "10.0.0.2" I know it is functionally immaterial, but boy howdy do I ever wish we'd chosen a better convention for how to refer to the relationship between these system components.

I know it is functionally immaterial There's nothing wrong with Master/slave. We also read man pages, shove male plugs into female ports, fork and kill children, and use dozens of other odd words to describe perfectly harmless things.

http://en.wikipedia.org/wiki/Deforestation_%28computer_scien...

Re: pg_rewind in PostgreSQL 9.5

#45

Post by dijit »

jeffdavis wrote: post #35
bradhe wrote: post #19

every step forward in modern relational DBs for reliability is a step backwards for operations and the simplicity of the model. If you're steeped in the ecosystem and know how things "used to be" you don't see how insane things actually are. Forrest, trees, etc.

Can you please expand?

he's scared of the increasing complexity of individual systems.

personally, I see where he's coming from with things like systemd.

complexity != usability.

simple things work well and work reliably.

however, I like the increasing features surrounding postgresql- and I'm playing devils advocate. (and anyway, it's not like it's part of the core build, the postgresql binary is untouched!)

Re: pg_rewind in PostgreSQL 9.5

#46

Post by waitwaitwhay »

rosser wrote: post #32
moe wrote: post #31

Earlier quoted context omitted.

Literally six lines Which is six lines too many for what is inherently a runtime configuration that shouldn't be mentioned in any config file at all. Then you use pg_basebackup, and start the slave. I take it you don't care for failover and monitoring then? Otherwise you would have mentioned either repmgr and pgpool here, or your conglomerate of rather complex homegrown scripts to mimic their functionality.

No, and nor did I mention "fencing". I said "minimum viable". No other open source RDBMS manages those things, so calling postgres out for lacking them is a little disingenuous. Replication isn't "inherently" anything. Making it a runtime thing is one way of doing it . Just because it's your preferred way (or — as often seems to be the case among people who share your opinion about how postgres does it — just how the…

> No other open source RDBMS manages those things, so calling postgres out for lacking them is a little disingenuous.

Doesnt MySQL Cluster do this pretty well?

Re: pg_rewind in PostgreSQL 9.5

#47

Post by moe »

lobster_johnson wrote: post #41
moe wrote: post #33

Earlier quoted context omitted.

fencing Which relates to a master/slave cluster as in how? no other open source RDBMS out there manages those things either Rethinkdb, Redis, Riak, ElasticSearch all do. (them not being relational is irrelevant to replication) Making it a runtime thing is one way of doing it. It's the only way of doing it for modern, dynamic application layouts (the whole "cloud" thing). The stereotypical static database cluster, car…

> Rethinkdb, Redis, Riak, ElasticSearch all do. None of those are RDBMSes. Since they all lack schemas, constraints, strict consistency and ACID transactions, their replication logic can be much simpler. There are some very good reasons that true multimaster replication is nearly unknown among relational databases.

Since they all lack schemas, constraints, strict consistency and ACID transactions, their replication logic can be much simpler.

Why would an RDBMS txlog be somehow harder to ship than any other db log?

And how is that an excuse for crappy usability in any case?

Re: pg_rewind in PostgreSQL 9.5

#48

Post by pyvpx »

moe wrote: post #39
ploxiln wrote: post #38

Earlier quoted context omitted.

Bringing up Redis, Riak, and ElasticSearch is nice... You see, ElasticSearch gets split brain surprisingly easily, if you have a real significant load / number of hosts / host failures you will most definitely experience it, whereas with most other databases it'll only happen once every couple of years to the very largest and heaviest users. It's also just not known for data consistency, you just have to be able to r…

Frankly these new-fangled auto-everything databases are untrustworthy I'm not asking for postgres to change its ways and become "untrustworthy". I'm merely asking that postgres wraps more automation and convenience around the things that it already does. Replication should generally be operated from inside psql. Spinning up a slave should be a single command. Failing over should be a single command. There should be m…

> I'm merely asking that postgres wraps more automation and convenience around the things that it already does.

and if history is any indication, it will when it's good and ready to be convenient and automated. as a previous poster (or two) most eloquently noted, postgres tends to work from the bottom up. when the "fundamentals" have been hashed out and thoroughly tested through all or nearly all edge cases, then convenient interfaces "bubble up"

and hey, your patches are always welcomed :)

Re: pg_rewind in PostgreSQL 9.5

#49

Post by moe »

lobster_johnson wrote: post #42
moe wrote: post #36

Earlier quoted context omitted.

Because when you promote a slave, you also need to fence writes to the master, otherwise you can end up in a "split-brain" scenario. There is only one master, hence there can not be a "split brain". You seem to be confusing master/slave and multi-master configuration (which pg doesn't support). I disagree. Their being relational imposes extra constraints upon replication. ACID? Well, it doesn't. Look, I'm rapidly los…

> There is only one master, hence there can not be a "split brain". That's very naive. There are situations -- very real ones -- where the failing master doesn't know it is failing, where clients can still connect and the new master can't safely tell the old one to stop being a master. This is where fencing comes in, and other concepts like heartbeats and STONITH and backup interconnects and so on, all of which makes…

There are situations

I'm tired of contrived "situations" to serve as lame excuses for usability stuck in the 80s.

Fencing is a red herring here, introduced by rosser to sound important. It's usually trivial to implement and completely unrelated to the problem that creating and in particular modifying a replica-pair is still a too complex, manual and error-prone operation in postgres.

That's one step before any "fencing" even comes into play.

Re: pg_rewind in PostgreSQL 9.5

#50

Post by moe »

pyvpx wrote: post #48
moe wrote: post #39

Earlier quoted context omitted.

Frankly these new-fangled auto-everything databases are untrustworthy I'm not asking for postgres to change its ways and become "untrustworthy". I'm merely asking that postgres wraps more automation and convenience around the things that it already does. Replication should generally be operated from inside psql. Spinning up a slave should be a single command. Failing over should be a single command. There should be m…

> I'm merely asking that postgres wraps more automation and convenience around the things that it already does. and if history is any indication, it will when it's good and ready to be convenient and automated. as a previous poster (or two) most eloquently noted, postgres tends to work from the bottom up. when the "fundamentals" have been hashed out and thoroughly tested through all or nearly all edge cases, then con…

and if history is any indication, it will

Very true. I was merely bemoaning the status quo. This is a thread about a third party tool to patch up a rather severe UX flaw after all...

Post reply on HN