Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

scylladb.com

Post reply on HN
Print view View original

11–20 of 99 posts

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#11

Post by acconsta »

whalesalad wrote: post #6
nattaylor wrote: post #4

>The Scylla design, right, is based on a modern shared-nothing approach. Scylla runs multiple engines, one per core, each with its own memory, CPU and multi-queue NIC. We can easily reach 1 million CQL operations on a single commodity server. In addition, Scylla targets consistent low latency, under 1 ms, for inserts, deletes, and reads. Interesting. From: http://www.scylladb.com/technology/architecture/

So on virtualized hardware, namely AWS, I'm sure the benchmarks won't be so magnificent. Needing a dedicated nic per core is a big deal unless you're at a pretty large scale.

How big do you have to be to lease hardware?

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#12

Post by jandrewrogers »

Very nice.

Broadly speaking, this is the correct style of architecture for a database engine on modern hardware. It is vastly more efficient in terms of throughput than the more traditional architectures common in open source. It also lends itself to elegant, compact implementations. I've been using similar architectures for several years now.

While I have not benchmarked their particular implementation, my first-hand experience is that these types of implementations are always at least 10x faster on the same hardware than a nominally equivalent open source database engine, so the performance claim is completely believable. One of my longstanding criticisms of open source data infrastructure has always been the very poor operational efficiency at a basic architectural level; many closed source companies have made a good business arbitraging the gross efficiency differences.

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#15

Post by jandrewrogers »

whalesalad wrote: post #6
nattaylor wrote: post #4

>The Scylla design, right, is based on a modern shared-nothing approach. Scylla runs multiple engines, one per core, each with its own memory, CPU and multi-queue NIC. We can easily reach 1 million CQL operations on a single commodity server. In addition, Scylla targets consistent low latency, under 1 ms, for inserts, deletes, and reads. Interesting. From: http://www.scylladb.com/technology/architecture/

So on virtualized hardware, namely AWS, I'm sure the benchmarks won't be so magnificent. Needing a dedicated nic per core is a big deal unless you're at a pretty large scale.

A modern Ethernet chipset has a large number of independent hardware queues. These can be assigned to VMs for direct access to the NIC, bypassing the hypervisor. AWS, since you used that example, offers instances with this type of direct bypass.

Just to pull an example from memory, the ubiquitous Intel 82599 10GbE NIC silicon has up to 128 TX and RX queues in hardware. IIRC, these are bundled in pairs for direct access in virtualized environments, so in principle you could have 64 virtual cores each with their own dedicated physical hardware queue. This is almost certainly what they were talking about. That is the whole point of this feature in Ethernet silicon; it gives cores (virtual or physical) dedicate network hardware off a single NIC.

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#18

Post by acconsta »

jandrewrogers wrote: post #12

Very nice. Broadly speaking, this is the correct style of architecture for a database engine on modern hardware. It is vastly more efficient in terms of throughput than the more traditional architectures common in open source. It also lends itself to elegant, compact implementations. I've been using similar architectures for several years now. While I have not benchmarked their particular implementation, my first-han…

Agreed, but which architectural features are you referring to?

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#19

Post by acconsta »

It's exciting to finally see this. Cassandra's strengths were in its distributed architecture (no master, tunable consistency, etc.). The database engine itself has always been a bit of a mess (https://issues.apache.org/jira/browse/CASSANDRA-8099).

Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster

#20

Post by z92 »

acconsta wrote: post #18
jandrewrogers wrote: post #12

Very nice. Broadly speaking, this is the correct style of architecture for a database engine on modern hardware. It is vastly more efficient in terms of throughput than the more traditional architectures common in open source. It also lends itself to elegant, compact implementations. I've been using similar architectures for several years now. While I have not benchmarked their particular implementation, my first-han…

Agreed, but which architectural features are you referring to?

Basically ditched Java in favor of C++, and used a C++ framework called Seastar.

"The Scylla design, right, is based on a modern shared-nothing approach. Scylla runs multiple engines, one per core, each with its own memory, CPU and multi-queue NIC."

http://www.scylladb.com/technology/architecture/

Post reply on HN