Live data from Hacker News

Jepsen: YugaByte DB 1.1.9

jepsen.io

11–20 of 49 posts

Re: Jepsen: YugaByte DB 1.1.9

#11
post #7
post #2

If they rely on clocks, why don't they use PTP? Am I missing something?

I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…

I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.

Re: Jepsen: YugaByte DB 1.1.9

#12
post #9
post #7

Earlier quoted context omitted.

I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…

My comment was that PTP is a much better protocol than NTP, and in the doc they only talk about NTP: https://docs.yugabyte.com/latest/deploy/checklist/#clock-syn... https://en.wikipedia.org/wiki/Precision_Time_Protocol

Hi @shin_lao

As @aphyr had mentioned, any NTP-alike system would work. We can update the docs to mention PTP, we do work with AWS Time Sync as well (which uses Chrony).

Re: Jepsen: YugaByte DB 1.1.9

#13
post #7

Earlier quoted context omitted.

I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…

I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.

In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible:

You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computation safely, though! For instance, you can't do something like a minimum-balance constraint, ensuring that an account always contains $25 or more, if you allow both deposits and withdrawals, in a commutative system. Why? Because order matters! Deposit, withdraw is different than withdraw, deposit, in terms of their intermediate states.

For order, you can use a consensus mechanism, like ZAB (Zookeeper), Paxos (Riak SC, Cassandra LWT), or Raft (etcd, consul) to replicate arbitrary state machines without any clock dependence at all. These systems require at least one round trip to establish consensus, and their guarantees only apply within the consensus system itself.

What if you have multiple consensus groups? Say, one per shard? Then you need a protocol to coordinate transactions on top of that. You can execute an atomic commit protocol for cross-shard transactions, perhaps using a consensus system. Or you can use a protocol like Calvin to obtain serializability (or stronger) across shards without relying on clocks. That's what FaunaDB does. That adds a round-trip, but if you're clever, you may only have to pay that round-trip cost between different datacenters once.

Another tactic is to exploit well-synchronized clocks to obtain consistent views across independent consensus groups. You can use this technique to (theoretically) reduce the number of round trips a transaction costs, and there are different ways to balance whether you pay increased latency on read or write transactions. Spanner, CockroachDB, and YugaByte DB all take this approach, with different tradeoffs.

Spanner is backed by custom hardware and carefully designed software, to obtain tight bounds on clock error. CockroachDB and YugaByte DB leave that problem to you, the operator.

Often, a database uses a stronger replication mechanism inside a datacenter, but when it comes to replicating between datacenters, backs off to a weaker strategy which doesn't offer the same safety invariants.

Re: Jepsen: YugaByte DB 1.1.9

#14
post #13

Earlier quoted context omitted.

I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.

In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…

Thanks, this is a really helpful overview!

Re: Jepsen: YugaByte DB 1.1.9

#15
post #13

Earlier quoted context omitted.

I'm not an expert, but don’t all modern databases suffer from these issues? I thought they all depended on clock sync like Google Spanner.

In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…

While FoundationDB uses Paxos for cluster state (like leader election), it is not on the commit path for a transaction. If any process fails in the transaction system (not storage processes), the cluster is reconfigured by the coordinators and every component is replaced. Transactions do not proceed during failures, but the cluster will replace the failed process in a few seconds and resume.

(This is not meant to be a contradiction, just pointing out an important difference compared to systems that allow progress in parallel with failures.)

Re: Jepsen: YugaByte DB 1.1.9

#16
post #13

Earlier quoted context omitted.

In short, no: many transactional databases don't rely on clocks for safety. I'm going to speak in broad terms here--there's a lot of nuance and special cases that we can dig into, but I'd like to keep this accessible: You can use CRDTs, and other commutative data structures, to obtain totally-available replicated objects across wide area networks. Systems like Riak do this. CRDTs can't express some types of computati…

While FoundationDB uses Paxos for cluster state (like leader election), it is not on the commit path for a transaction. If any process fails in the transaction system (not storage processes), the cluster is reconfigured by the coordinators and every component is replaced. Transactions do not proceed during failures, but the cluster will replace the failed process in a few seconds and resume. (This is not meant to be…

Ah, my mistake, thank you! It's been a long while since I talked to the FDB folks and I guess I misunderstood where they were using Paxos!

Re: Jepsen: YugaByte DB 1.1.9

#17

Is it also tested against ScyllaDB ? ScyllaDB could be up to 10x performant than Cassandra as backend storage.

Yugabyte doesn't use Cassandra, it's a custom-built database using RocksDB as a key/value layer and with an internal document-store representation.

It offers access to the data in multiple interfaces: Redis, Cassandra CQL and now PostgreSQL.

Re: Jepsen: YugaByte DB 1.1.9

#18
post #3

First time I heard about them. Seems to be another distributed SQL (aka 'newsql') alternative to TiDB and CockroachDB. Based on RocksDB (like Cockroach) with a custom distributed key/val layer and and additional SQL layer on top. PostgreSQL protocol compatible. OS with Apache license. Seems interesting. (when ignoring the "planet scale SQL" marketing speak... [1]) [1] https://www.yugabyte.com/planet-scale-sql/

As an expert in the DB space, I'm extraordinarily cynical.

But their willingness to license it as truly open, Apache-style, instantly is a big win-over.

I'm a competitor, but I can tell these guys/gals are genuine in their efforts.

We need more people, teams, and DBs like YugaByteDB in the world.

Thank you for your efforts.

Re: Jepsen: YugaByte DB 1.1.9

#19
Since it doesn't support serializable transactions I'm not sure why FoundationDB would be mentioned as a comparison in the write up. The operations it does support seem to set the bar pretty low as to what to test.

edit: good reply by the founder of YugaByte but for some reason the comment is dead. I have noticed that when founders don't have an account on here and then something comes up where they need to reply their comments are often deaded.

Re: Jepsen: YugaByte DB 1.1.9

#20
post #3

First time I heard about them. Seems to be another distributed SQL (aka 'newsql') alternative to TiDB and CockroachDB. Based on RocksDB (like Cockroach) with a custom distributed key/val layer and and additional SQL layer on top. PostgreSQL protocol compatible. OS with Apache license. Seems interesting. (when ignoring the "planet scale SQL" marketing speak... [1]) [1] https://www.yugabyte.com/planet-scale-sql/

As an expert in the DB space, I'm extraordinarily cynical. But their willingness to license it as truly open, Apache-style, instantly is a big win-over. I'm a competitor, but I can tell these guys/gals are genuine in their efforts. We need more people, teams, and DBs like YugaByteDB in the world. Thank you for your efforts.

The question is, how many of these very similar databases can the market support?

The field is getting crowded, and the database market is already quite competitive as it is, without these new competitors.

There just are not that many use cases where a larger Postgres/MySQL instance with one or two replicas is insufficient.

From a user perspective, I'd much rather have one or two successful companies where I can be reasonably certain that the product will be maintained in 5 years than too much competition.

Post reply on HN