Jepsen: YugaByte DB 1.1.9
jepsen.io
Jepsen: YugaByte DB 1.1.9
1–10 of 49 posts
Re: Jepsen: YugaByte DB 1.1.9
#2If they rely on clocks, why don't they use PTP? Am I missing something?
Re: Jepsen: YugaByte DB 1.1.9
#3First time I heard about them.
Seems to be another distributed SQL (aka 'newsql') alternative to TiDB and CockroachDB.
Based on RocksDB (like Cockroach) with a custom distributed key/val layer and and additional SQL layer on top. PostgreSQL protocol compatible.
OS with Apache license.
Seems interesting. (when ignoring the "planet scale SQL" marketing speak... [1])
Re: Jepsen: YugaByte DB 1.1.9
#4Is it also tested against ScyllaDB ?
ScyllaDB could be up to 10x performant than Cassandra as backend storage.
Re: Jepsen: YugaByte DB 1.1.9
#5Is it also tested against ScyllaDB ? ScyllaDB could be up to 10x performant than Cassandra as backend storage.
Jepsen is not a performance test; we verify safety. I haven't looked at ScyllaDB personally, but you can read about Scylla's own work testing their database here [1], and see some of the issues they found here [2].
[1]: https://www.scylladb.com/2016/02/11/jepsen-testing/
[2]: https://github.com/scylladb/scylla/issues?utf8=%E2%9C%93&q=i...
Re: Jepsen: YugaByte DB 1.1.9
#6Is it also tested against ScyllaDB ? ScyllaDB could be up to 10x performant than Cassandra as backend storage.
Jepsen is not a performance test; we verify safety. I haven't looked at ScyllaDB personally, but you can read about Scylla's own work testing their database here [1], and see some of the issues they found here [2]. [1]: https://www.scylladb.com/2016/02/11/jepsen-testing/ [2]: https://github.com/scylladb/scylla/issues?utf8=%E2%9C%93&q=i...
I never said Jespen is about performance.
My comment was only about considering ScyllaDB as backend choice.
Thanks for the links anyway.
Re: Jepsen: YugaByte DB 1.1.9
#7If they rely on clocks, why don't they use PTP? Am I missing something?
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in theory, datacenter and hardware providers could do better than pure-software time synchronization by, say, offering dedicated physical links to a local atomic + GPS clock ensemble. AWS TimeSync is a step in this direction, and I wouldn't be surprised if we see more accurate clocks in the future.
There are still tons of caveats with this idea--Linux and most database software ain't realtime, for starters--but you can imagine a world in which clock errors are sufficiently bounded and infrequent that they no longer represent the most urgent threat to safety. That's ultimately a quantitative risk assessment.
My suspicion is that DB vendors like YugaByte and CockroachDB are making a strategic bet that although clocks right now are pretty terrible, they won't be that way forever. I'd like to see more rigorous measurement on this front, because while I've got plenty of anecdotes, I don't think we have a broad statistical picture of how bad typical clocks are, and whether they're improving.
Re: Jepsen: YugaByte DB 1.1.9
#8Earlier quoted context omitted.
Jepsen is not a performance test; we verify safety. I haven't looked at ScyllaDB personally, but you can read about Scylla's own work testing their database here [1], and see some of the issues they found here [2]. [1]: https://www.scylladb.com/2016/02/11/jepsen-testing/ [2]: https://github.com/scylladb/scylla/issues?utf8=%E2%9C%93&q=i...
I never said Jespen is about performance. My comment was only about considering ScyllaDB as backend choice. Thanks for the links anyway.
YugaByte product manager here. The YCQL API which passes Jepsen has its roots in Cassandra Query Language but does not use Cassandra as its backend store. It’s backend store is DocDB, which is a Google Spanner-inspired distributed document store.
Re: Jepsen: YugaByte DB 1.1.9
#9If they rely on clocks, why don't they use PTP? Am I missing something?
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…
My comment was that PTP is a much better protocol than NTP, and in the doc they only talk about NTP:
https://docs.yugabyte.com/latest/deploy/checklist/#clock-syn...
Re: Jepsen: YugaByte DB 1.1.9
#10Earlier quoted context omitted.
I think the YB team members are probably best equipped to talk about this, but I can note that while some databases do build their own clock synchronization protocol, many prefer to let the OS handle clocks. For one thing, clock sync is surprisingly tricky to do well, so it makes sense to write daemons that do it well once and be able to re-use them in lots of contexts. There's also the question of HW support: in the…
My comment was that PTP is a much better protocol than NTP, and in the doc they only talk about NTP: https://docs.yugabyte.com/latest/deploy/checklist/#clock-syn... https://en.wikipedia.org/wiki/Precision_Time_Protocol
Yeah, you can deploy any type of NTP-alike time synchronization you like. That could be ntpd, chrony, hypervisor-guest synchronization tools, etc etc.