If they rely on clocks, why don't they use PTP? Am I missing something?
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…
Jepsen: YugaByte DB 1.1.9
11–20 of 49 posts
Re: Jepsen: YugaByte DB 1.1.9
#12Earlier quoted context omitted.
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…
My comment was that PTP is a much better protocol than NTP, and in the doc they only talk about NTP: https://docs.yugabyte.com/latest/deploy/checklist/#clock-syn... https://en.wikipedia.org/wiki/Precision_Time_Protocol
As @aphyr had mentioned, any NTP-alike system would work. We can update the docs to mention PTP, we do work with AWS Time Sync as well (which uses Chrony).
Re: Jepsen: YugaByte DB 1.1.9
#13Earlier quoted context omitted.
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…
I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.
You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computation safely, though! For instance, you can't do something like a minimum-balance constraint, ensuring that an account always contains $25 or more, if you allow both deposits and withdrawals, in a commutative system. Why? Because order matters! Deposit, withdraw is different than withdraw, deposit, in terms of their intermediate states.
For order, you can use a consensus mechanism, like ZAB (Zookeeper), Paxos (Riak SC, Cassandra LWT), or Raft (etcd, consul) to replicate arbitrary state machines without any clock dependence at all. These systems require at least one round trip to establish consensus, and their guarantees only apply within the consensus system itself.
What if you have multiple consensus groups? Say, one per shard? Then you need a protocol to coordinate transactions on top of that. You can execute an atomic commit protocol for cross-shard transactions, perhaps using a consensus system. Or you can use a protocol like Calvin to obtain serializability (or stronger) across shards without relying on clocks. That's what FaunaDB does. That adds a round-trip, but if you're clever, you may only have to pay that round-trip cost between different datacenters once.
Another tactic is to exploit well-synchronized clocks to obtain consistent views across independent consensus groups. You can use this technique to (theoretically) reduce the number of round trips a transaction costs, and there are different ways to balance whether you pay increased latency on read or write transactions. Spanner, CockroachDB, and YugaByte DB all take this approach, with different tradeoffs.
Spanner is backed by custom hardware and carefully designed software, to obtain tight bounds on clock error. CockroachDB and YugaByte DB leave that problem to you, the operator.
Often, a database uses a stronger replication mechanism inside a datacenter, but when it comes to replicating between datacenters, backs off to a weaker strategy which doesn't offer the same safety invariants.
Re: Jepsen: YugaByte DB 1.1.9
#14Earlier quoted context omitted.
I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.
In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…
Re: Jepsen: YugaByte DB 1.1.9
#15Earlier quoted context omitted.
I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.
In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…
(This is not meant to be a contradiction, just pointing out an important difference compared to systems that allow progress in parallel with failures.)
Re: Jepsen: YugaByte DB 1.1.9
#16Earlier quoted context omitted.
In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…
While FoundationDB uses Paxos for cluster state (like leader election), it is not on the commit path for a transaction. If any process fails in the transaction system (not storage processes), the cluster is reconfigured by the coordinators and every component is replaced. Transactions do not proceed during failures, but the cluster will replace the failed process in a few seconds and resume. (This is not meant to be…
Re: Jepsen: YugaByte DB 1.1.9
#17Is it also tested against ScyllaDB ? ScyllaDB could be up to 10x performant than Cassandra as backend storage.
It offers access to the data in multiple interfaces: Redis, Cassandra CQL and now PostgreSQL.
Re: Jepsen: YugaByte DB 1.1.9
#18First time I heard about them. Seems to be another distributed SQL (aka 'newsql') alternative to TiDB and CockroachDB. Based on RocksDB (like Cockroach) with a custom distributed key/val layer and and additional SQL layer on top. PostgreSQL protocol compatible. OS with Apache license. Seems interesting. (when ignoring the "planet scale SQL" marketing speak... [1]) [1] https://www.yugabyte.com/planet-scale-sql/
But their willingness to license it as truly open, Apache-style, instantly is a big win-over.
I'm a competitor, but I can tell these guys/gals are genuine in their efforts.
We need more people, teams, and DBs like YugaByteDB in the world.
Thank you for your efforts.
Re: Jepsen: YugaByte DB 1.1.9
#19edit: good reply by the founder of YugaByte but for some reason the comment is dead. I have noticed that when founders don't have an account on here and then something comes up where they need to reply their comments are often deaded.
Re: Jepsen: YugaByte DB 1.1.9
#20First time I heard about them. Seems to be another distributed SQL (aka 'newsql') alternative to TiDB and CockroachDB. Based on RocksDB (like Cockroach) with a custom distributed key/val layer and and additional SQL layer on top. PostgreSQL protocol compatible. OS with Apache license. Seems interesting. (when ignoring the "planet scale SQL" marketing speak... [1]) [1] https://www.yugabyte.com/planet-scale-sql/
As an expert in the DB space, I'm extraordinarily cynical. But their willingness to license it as truly open, Apache-style, instantly is a big win-over. I'm a competitor, but I can tell these guys/gals are genuine in their efforts. We need more people, teams, and DBs like YugaByteDB in the world. Thank you for your efforts.
The field is getting crowded, and the database market is already quite competitive as it is, without these new competitors.
There just are not that many use cases where a larger Postgres/MySQL instance with one or two replicas is insufficient.
From a user perspective, I'd much rather have one or two successful companies where I can be reasonably certain that the product will be maintained in 5 years than too much competition.