The license is Affero GPL, which means you need to open-source your code even if you use it for a service. "Traditional" GPL was effective only while redistributing. That means you would need to go for commercial license whenever you build a service on it. Which in fact is a fair approach for a business model when there is a company behind an open source project. Especially that this time there is no lock-in. You could always come back to Cassandra.
ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
21–30 of 99 posts
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#22will this be the next docker in the nosql database?
I don't even see any connection how could this be the next docker in the nosql world... in other words i didn't get at all what you mean...
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#23Very nice. Broadly speaking, this is the correct style of architecture for a database engine on modern hardware. It is vastly more efficient in terms of throughput than the more traditional architectures common in open source. It also lends itself to elegant, compact implementations. I've been using similar architectures for several years now. While I have not benchmarked their particular implementation, my first-han…
I agree as well, on the networking aspect since based on their diagram they are utilizing Intel NICs which Intel provides this DPDK to bypass the kernel space and access the hardware from the application itself.
Now my question is how portable Scylla be in terms of NIC vendors?
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#24Very nice. Broadly speaking, this is the correct style of architecture for a database engine on modern hardware. It is vastly more efficient in terms of throughput than the more traditional architectures common in open source. It also lends itself to elegant, compact implementations. I've been using similar architectures for several years now. While I have not benchmarked their particular implementation, my first-han…
Agreed, but which architectural features are you referring to?
Over the last decade, the distributed system nature of modern server hardware internals has become painfully evident in how software architectures scale on a single machine. The traditional approaches -- multithreading, locking, lock-free structures, etc -- are all forms of coordination and agreement in a distributed system, with the attendant scalability problems if not used very carefully.
At some point several years ago, a few people noticed that if you attack the problem of scalable distribution within a single server the same way you would in large distributed systems (e.g. shared nothing architectures) that you could realize huge performance increases on a single machine. The caveat is that the software architectures look unorthodox.
The general model looks like this:
- one process per core, each locked to a single core
- use locked local RAM only (effectively limiting NUMA)
- direct dedicated network queue (bypass kernel)
- direct storage I/O (bypass kernel)
If you do it right, you minimize the amount of silicon that is shared between processes which has surprisingly large performance benefits. Linux has facilities that make this relatively straightforward too.
As a consequence, adjacent cores on the same CPU have only marginally more interaction with each other than cores on different machines entirely. Treating a single server as a distributed cluster of 1-core machines, and writing the software in such a way that the operating system behavior reflects that model to the extent possible, is a great architecture for extreme performance but you rarely see it outside of closed source software.
As a corollary, garbage-collected languages do not work for this at all.
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#25Finally, back to sanity of great old-school products, like Informix, by dropping Java (the whole scam) for C++14 and by paying attention to details of an underlying OS (again). Same trend, by the way, is in Android development.
10x speedup (same algorithms, same architecture) replacing Java with C++ is not possible (~2x at max).
One of the latest benchmarks I've seen is "Comparison of Programming Languages in Economics" [1] for code without any IO just number crunching, has a 1.91 to 2.69 speedup of using C++ compared to Java. So any code involving IO is going to be slower.
Replacing bad Java code with excellent machine aligned C++ a 10x speedup is possible.
[1] https://github.com/jesusfv/Comparison-Programming-Languages-...
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#26Finally, back to sanity of great old-school products, like Informix, by dropping Java (the whole scam) for C++14 and by paying attention to details of an underlying OS (again). Same trend, by the way, is in Android development.
10x speedup (same algorithms, same architecture) replacing Java with C++ is not possible (~2x at max). One of the latest benchmarks I've seen is "Comparison of Programming Languages in Economics" [1] for code without any IO just number crunching, has a 1.91 to 2.69 speedup of using C++ compared to Java. So any code involving IO is going to be slower. Replacing bad Java code with excellent machine aligned C++ a 10x sp…
It's particularly flawed given:
a) IO is such a large portion of the problem b) Hypertable isn't just way, way faster.
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#27Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#28Earlier quoted context omitted.
Agreed, but which architectural features are you referring to?
Over the last decade, the distributed system nature of modern server hardware internals has become painfully evident in how software architectures scale on a single machine. The traditional approaches -- multithreading, locking, lock-free structures, etc -- are all forms of coordination and agreement in a distributed system, with the attendant scalability problems if not used very carefully. At some point several yea…
I really appreciate your insight, but I didn't understand why you keep implying "open source" as some kind of lack-behind design?
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#29Earlier quoted context omitted.
So on virtualized hardware, namely AWS, I'm sure the benchmarks won't be so magnificent. Needing a dedicated nic per core is a big deal unless you're at a pretty large scale.
How big do you have to be to lease hardware?
leasing is often cheaper than the alternatives
Re: ScyllaDB: Drop-in replacement for Cassandra that claims to be 10x faster
#30Earlier quoted context omitted.
Agreed, but which architectural features are you referring to?
Over the last decade, the distributed system nature of modern server hardware internals has become painfully evident in how software architectures scale on a single machine. The traditional approaches -- multithreading, locking, lock-free structures, etc -- are all forms of coordination and agreement in a distributed system, with the attendant scalability problems if not used very carefully. At some point several yea…
Strongly agreed. It's great that these HPC techniques are finally starting to trickle down.