120K distributed consistent writes per second with Calvin • Hacker News Board

Re: 120K distributed consistent writes per second with Calvin

#61

Post by g0del_was_wr0ng » Sat, Feb 25, 2017, 3:50 PM UTC

evanweaver wrote: post #60

g0del_was_wr0ng wrote: post #59

Earlier quoted context omitted.

It's doing 12k rows in 3k user-issued write operations/transaction per second. Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores. 12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it…

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only. We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

Re: 120K distributed consistent writes per second with Calvin

#62

Post by evanweaver » Sat, Feb 25, 2017, 3:53 PM UTC

evanweaver wrote: post #60

g0del_was_wr0ng wrote: post #59

Earlier quoted context omitted.

It's doing 12k rows in 3k user-issued write operations/transaction per second. Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores. 12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it…

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only. We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

Re: 120K distributed consistent writes per second with Calvin

#63

Post by g0del_was_wr0ng » Sat, Feb 25, 2017, 4:01 PM UTC

evanweaver wrote: post #62

evanweaver wrote: post #60

Earlier quoted context omitted.

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only. We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

(An uninitiated reader would assume you're comitting 120k rows per second from the title, whereas it's "only" 12k rows and "only" 3k actual operations over the wire. Still, 3-12k is pretty impressive)

Re: 120K distributed consistent writes per second with Calvin

#64

Post by qaq » Sat, Feb 25, 2017, 4:14 PM UTC

g0del_was_wr0ng wrote: post #55

qaq wrote: post #54

Earlier quoted context omitted.

By extrapolation writes are IO bound you don't need crazy expensive things to get to the needed number of IOPS Intel 750 is 230,000 random writes @ $320 per PCI-E SSD. 9 drive config is over 2,000,000 IOPS for less than 3K.

Yeah it's a bit more complex than that... The disk "IOPS" number on the box doesn't translate 1:1 or even linearly to number of committed durable transactions per second. You should try this with postgres and see how it goes.

If you land me 30K for the hardware I would be glad to do it :). For writing WAL it scales reasonably close to linear

Re: 120K distributed consistent writes per second with Calvin

#65

Post by qaq » Sat, Feb 25, 2017, 4:17 PM UTC

g0del_was_wr0ng wrote: post #56

pg314 wrote: post #49

Earlier quoted context omitted.

Like others have commented, those numbers seem to be too high for writes. On the other hand, the Fauna numbers don't seem that impressive to me. On a mid-2011 Macbook Air, I get 2600 transactions per second (read-committed) in PostgreSQL 9.6. Setup is as follows: CREATE TABLE IF NOT EXISTS foo(a TEXT, b TEXT, c TEXT, d TEXT); CREATE INDEX IF NOT EXISTS idx_foo_a ON foo(a); CREATE INDEX IF NOT EXISTS idx_foo_b ON foo(…

Well, you're missing that in Faunas case the writes are durably stored on N machines. I.e. their system provides fault tolerance in case a machine fails. You can't really do the same thing with postgres (without trading off full ACID compliance).

you sure can get reasonable close with 2 phase commit, and starting with next release and quorum commit you will be able to do exactly that.

Re: 120K distributed consistent writes per second with Calvin

#66

Post by evanweaver » Sat, Feb 25, 2017, 4:23 PM UTC

qaq wrote: post #65

g0del_was_wr0ng wrote: post #56

Earlier quoted context omitted.

Well, you're missing that in Faunas case the writes are durably stored on N machines. I.e. their system provides fault tolerance in case a machine fails. You can't really do the same thing with postgres (without trading off full ACID compliance).

you sure can get reasonable close with 2 phase commit, and starting with next release and quorum commit you will be able to do exactly that.

Quorum commit is not the same as distributed strict serialization because replicas can desynchronize.

Additionally you have the bottleneck of a single master. Will it be possible to do a quorum read as well, and will every transaction on the master be doing it? Then you are starting to get closer, although many anomalies are still possible.

Re: 120K distributed consistent writes per second with Calvin

#67

Post by lobster_johnson » Sat, Feb 25, 2017, 7:28 PM UTC

wwilson wrote: post #33

atombender wrote: post #31

Earlier quoted context omitted.

What do you guys think about TiDB and CockroachDB, both of which are SQL layers on top of a distributed K/V store?

Full disclosure: I now work at Google on Cloud Spanner which competes with both products you mentioned. These are just my personal (and probably highly biased) opinions. I have some concerns about CockroachDB on both the performance and the reliability fronts. But I hugely admire what they're trying to do and I've heard that they're rapidly improving in both areas. TiDB is an exciting project that I've heard great th…

Agreed, Postgres does scale very well for non-Google-sized apps, though a lingering issue is handling failover.

But if one does need a bit more horizontal scalability, there don't seem to be a lot of options if you also want atomic, transactional updates (though not necessarily strict transaction isolation). I have an app that is conceptually a versioned document store, where each document is the sum of all its "patches"; when you submit a batch of patches, the rule is that these are applied atomically, and that the latest version of document thereafter reflects your patch (optimistic locking and retries take care of serialization and concurrent conflicts). I'm using PostgreSQL right now, which does this beautifully, but with limited scalability. I've looked for a better option, but not come up with anything.

Redis would handle this, but it would work purely thanks to single-threaded; and I don't feel like Redis is safe as a primary data store for anything except caches and such. Cassandra might do it, using atomic batches, although its lack of isolation could be awkward to work around.

Re: 120K distributed consistent writes per second with Calvin

#68

Post by pg314 » Sat, Feb 25, 2017, 8:03 PM UTC

g0del_was_wr0ng wrote: post #56

pg314 wrote: post #49

Earlier quoted context omitted.

Like others have commented, those numbers seem to be too high for writes. On the other hand, the Fauna numbers don't seem that impressive to me. On a mid-2011 Macbook Air, I get 2600 transactions per second (read-committed) in PostgreSQL 9.6. Setup is as follows: CREATE TABLE IF NOT EXISTS foo(a TEXT, b TEXT, c TEXT, d TEXT); CREATE INDEX IF NOT EXISTS idx_foo_a ON foo(a); CREATE INDEX IF NOT EXISTS idx_foo_b ON foo(…

Well, you're missing that in Faunas case the writes are durably stored on N machines. I.e. their system provides fault tolerance in case a machine fails. You can't really do the same thing with postgres (without trading off full ACID compliance).

True. Thanks for pointing that out. I think they probably should be putting the emphasis in their marketing on the fault tolerance i.s.o. the performance.

Re: 120K distributed consistent writes per second with Calvin

#69

Post by cakoose » Sat, Feb 25, 2017, 11:01 PM UTC

freels wrote: post #21

cakoose wrote: post #13

Earlier quoted context omitted.

How would you perform the classic "transfer money from one account to another" operation? Would you create a single operation that reads one record, checks that it's enough, then adds the amount to another record? Or maybe you'd first read both accounts, then issue a conditional write operation that makes sure the data hasn't changed before doing the write?

FaunaDB's query language makes it straightforward to do it the first way. All queries are serializable, so any preconditions checked would gate transaction commit as you would expect, and read-modify-write style transactions work. edit: here's an example in our Scala DSL: Let { val amount = 50 val balanceA = Select("data" / "balance", Get(Ref("accountA")) val balanceB = Select("data" / "balance", Get(Ref("accountB"))…

Is the wire format roughly isomorphic to the structure above? Or does the Scala library convert this code-like structure into something simpler/flatter?

(BTW, would be nice if I could read your API docs without signing up for an account.)

Re: 120K distributed consistent writes per second with Calvin

#70

Post by evanweaver » Sat, Feb 25, 2017, 11:31 PM UTC

cakoose wrote: post #69

freels wrote: post #21

Earlier quoted context omitted.

FaunaDB's query language makes it straightforward to do it the first way. All queries are serializable, so any preconditions checked would gate transaction commit as you would expect, and read-modify-write style transactions work. edit: here's an example in our Scala DSL: Let { val amount = 50 val balanceA = Select("data" / "balance", Get(Ref("accountA")) val balanceB = Select("data" / "balance", Get(Ref("accountB"))…

Is the wire format roughly isomorphic to the structure above? Or does the Scala library convert this code-like structure into something simpler/flatter? (BTW, would be nice if I could read your API docs without signing up for an account.)

It is isomorphic; right now it's layered onto JSON, but eventually we will support CBOR on the wire as well. Internally everything is CBOR with LZ4 block compression.

The docs will eventually be available without an account.