TAPIR: A new high-performance, transactional key-value store • Hacker News Board

Re: TAPIR: A new high-performance, transactional key-value store

#11

Post by kluck » Thu, Jan 14, 2016, 10:37 AM UTC

jph wrote: post #5

The key insight: "... existing transactional storage systems waste work and performance by incorporating a distributed transaction protocol and a replication protocol that both enforce strong consistency. Instead, we show that it is possible to provide distributed transactions with better performance and the same transaction and consistency model using replication with no consistency." The full paper is here: http://…

Interesting. Can you elaborate on this? I am not sure what "inconsistent replication" means.

Re: TAPIR: A new high-performance, transactional key-value store

#12

Post by mhw » Thu, Jan 14, 2016, 10:45 AM UTC

proksoup wrote: post #2

I found this video a couple links in, the speaker I think is Irene Zhang, listed for many of the commits on that repo: https://www.youtube.com/watch?v=yE3eMxYJDiE

And also the poster of this item, presumably.

Re: TAPIR: A new high-performance, transactional key-value store

#13

Post by ComNik » Thu, Jan 14, 2016, 11:21 AM UTC

kluck wrote: post #11

jph wrote: post #5

The key insight: "... existing transactional storage systems waste work and performance by incorporating a distributed transaction protocol and a replication protocol that both enforce strong consistency. Instead, we show that it is possible to provide distributed transactions with better performance and the same transaction and consistency model using replication with no consistency." The full paper is here: http://…

Interesting. Can you elaborate on this? I am not sure what "inconsistent replication" means.

It means that operations don't get executed in the same order on every replica. If the systems state depends on the ordering of operations, then each replica might be in a different state for the same set of operations.

"Consistent replication" would be using a protocol like Paxos to have the replicas decide on a single order of operations.

Re: TAPIR: A new high-performance, transactional key-value store

#14

Post by pjc50 » Thu, Jan 14, 2016, 11:39 AM UTC

jph wrote: post #5

The key insight: "... existing transactional storage systems waste work and performance by incorporating a distributed transaction protocol and a replication protocol that both enforce strong consistency. Instead, we show that it is possible to provide distributed transactions with better performance and the same transaction and consistency model using replication with no consistency." The full paper is here: http://…

"An error occurred while processing your request"

Re: TAPIR: A new high-performance, transactional key-value store

#15

Post by amelius » Thu, Jan 14, 2016, 12:24 PM UTC

ComNik wrote: post #13

kluck wrote: post #11

Earlier quoted context omitted.

Interesting. Can you elaborate on this? I am not sure what "inconsistent replication" means.

It means that operations don't get executed in the same order on every replica. If the systems state depends on the ordering of operations, then each replica might be in a different state for the same set of operations. "Consistent replication" would be using a protocol like Paxos to have the replicas decide on a single order of operations.

> If the systems state depends on the ordering of operations, then each replica might be in a different state for the same set of operations.

Which is almost always the case. Except of course, if you build your data as a growing "collection of knowledge", where the order of facts doesn't matter. But this is cheating, since you're implicitly bolting an ordering-mechanism on top of the system in this case.

Re: TAPIR: A new high-performance, transactional key-value store

#16

Post by tremon » Thu, Jan 14, 2016, 12:27 PM UTC

proksoup wrote: post #2

I found this video a couple links in, the speaker I think is Irene Zhang, listed for many of the commits on that repo: https://www.youtube.com/watch?v=yE3eMxYJDiE

It would be helpful to mention that the talk is about this topic as well, not just who's speaking.

Re: TAPIR: A new high-performance, transactional key-value store

#17

Post by TallGuyShort » Thu, Jan 14, 2016, 4:01 PM UTC

ComNik wrote: post #13

kluck wrote: post #11

Earlier quoted context omitted.

Interesting. Can you elaborate on this? I am not sure what "inconsistent replication" means.

It means that operations don't get executed in the same order on every replica. If the systems state depends on the ordering of operations, then each replica might be in a different state for the same set of operations. "Consistent replication" would be using a protocol like Paxos to have the replicas decide on a single order of operations.

Huh... isn't that required for being "transactional"? I should probably read the paper...

Re: TAPIR: A new high-performance, transactional key-value store

#18

Post by msackman » Thu, Jan 14, 2016, 4:05 PM UTC

nickpsecurity wrote: post #4

"enabling TAPIR to provide the same transaction model and consistency guarantees as existing systems, like Spanner, with better latency and throughput." Oh, hell yeah! Now that's great stuff. Can't wait to see the next step done by this or another team: building an alternative to the F1 RDBMS that Google built on Spanner. Would give CochroachDB some competition.

I'm the author of GoshawkDB.

I've just been watching the talk on this at https://www.youtube.com/watch?v=yE3eMxYJDiE. GoshawkDB has a very similar design wrt the messaging and replication design. In fact, in some places, it appears that GoshawkDB's design is a little simpler.

There are obviously many differences too: for example GoshawkDB runs transactions on the client, GoshawkDB uses Paxos Synod instead of 2PC, and GoshawkDB clients only connect to one server so there are 2 extra hops, but that's a constant so from a scaling pov, it should behave the same.

One of the biggest differences is GoshawkDB uses Vector Clocks (that can grow and shrink) rather than loosely synchronized clocks.

This TAPIR work does look great - I had no idea that it was ongoing. I'll read through the paper too, but it's great that GoshawkDB has so many design ideas in common.

Re: TAPIR: A new high-performance, transactional key-value store

#19

Post by imaginenore » Thu, Jan 14, 2016, 4:10 PM UTC

You claim it's high-performance, and you provide no benchmarks.

How does it compare with Redis, Aerospike, Tarantool, Couchbase/MemBase, Memacached, VoltDB, LevelDB, Kyoto Cabinet, Riak, Cassandra, RocksDB, LMDB, Neo4j, HBase, ArangoDB, Voldemont, FoundationDB?

Re: TAPIR: A new high-performance, transactional key-value store

#20

Post by dsp1234 » Thu, Jan 14, 2016, 4:20 PM UTC

kluck wrote: post #11

jph wrote: post #5

The key insight: "... existing transactional storage systems waste work and performance by incorporating a distributed transaction protocol and a replication protocol that both enforce strong consistency. Instead, we show that it is possible to provide distributed transactions with better performance and the same transaction and consistency model using replication with no consistency." The full paper is here: http://…

Interesting. Can you elaborate on this? I am not sure what "inconsistent replication" means.

An analogy that immediately comes to mind is the difference between running a vpn in 'tcp over tcp' mode vs 'tcp over udp' mode.

tcp itself is a reliable transport over unreliable media. So running tcp on top of tcp means running two sets of algorithms for reliability, ultimately doing more work than is needed. Running tcp over udp (where udp is unreliable) means you still get the reliability over the tcp overlay, but don't need to be worried about the udp layer since it can fail and the tcp overlay algorithms will fix up the data stream.