Scylla – Real-Time Big Data Database
scylladb.com
Scylla – Real-Time Big Data Database
1–10 of 53 posts
Re: Scylla – Real-Time Big Data Database
#2I posted this because I'm interested to hear from anyone using it - how has it worked out for you?
I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang.
Interesting as well is is AGPL - licensing is always contentious:
Re: Scylla – Real-Time Big Data Database
#3I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
We're using it with several customers: fast, reliable, straightforward.
Re: Scylla – Real-Time Big Data Database
#4I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
I think the main reason it's in C++ is because of its async executor, Seastar. There's a similar Rust project called Glommio but seems still very early.
Re: Scylla – Real-Time Big Data Database
#5I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
Only on Hackernews would somebody be surprised that high-performance system software would be written in C++...
Re: Scylla – Real-Time Big Data Database
#6The benchmarks against DynamoDB, Bigtable, & CockroachDB [1] appear quite impressive - anyone have real world experience that can attest to these claims of improved performance and reduced cost?
> Scylla vs DynamoDB – Database Benchmark
> 20x better throughput in the hot-partition test
> Scylla Cloud is 1/7 the expense of DynamoDB when running equivalent workloads
> Scylla Cloud: Average replication latency of 82ms. DynamoDB: Average latency of 370ms.
> Scylla vs Bigtable – Database Benchmark
> Scylla Cloud performs 26X better than Google Cloud Bigtable when applied with real-world, unoptimized data distribution
> Google BigTable requires 10X as many nodes to accept the same workload as Scylla Cloud
> Scylla Cloud was able to sustain 26x the throughput, and with read latencies 1/800th and write latencies less than 1/100th of Cloud Bigtable
> Scylla vs CockroachDB – Database Benchmark
> Loading 10x the data into Scylla took less than half the time it took for CockroachDB to load the much lesser dataset.
> Scylla handled 10x the amount of data.
> Scylla achieved 9.3x the throughput of CockroachDB at 1/4th the latency.
Re: Scylla – Real-Time Big Data Database
#7I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
Quoting the interview with ScyllaDB CTO, Avi Kivity ( https://www.scylladb.com/2020/06/30/ask-me-anything-with-avi... )
> Q: Would you implement Scylla in Go, Rust or Javascript if you could?
> Avi: Good question. I wouldn’t implement Scylla in Javascript. It’s not really a high-performance language, but I will note that Node.js and Seastar share many characteristics. Both are using a reactor pattern and designed for high concurrency. Of course the performance is going to be very different between the two, but writing code for Node.js and writing code for Seastar is quite similar.
> Go also has an interesting take on concurrency. I still wouldn’t use it for something like Scylla. It is a garbage-collected language so you lose a lot of predictability, and you lose some performance. The concurrency model is great. The language lacks generics. I like generics a lot and I think they are required for complex software. I also hear that Go is getting generics in the next iteration. Go is actually quite close to being useful for writing a high-performance database. It still has the downside of having a garbage collector, so from that point-of-view I wouldn’t pick it.
> If you are familiar with how Scylla uses the direct I/O and asynchronous I/O, this is not something that Go is great at right now. I imagine that it will evolve. So I wouldn’t pick Javascript or Go.
> However, the other language you mentioned, Rust, does have all of the correct characteristics that Scylla requires. Precise control over what happens. It doesn’t have a garbage collector so it means that you have predictability over how much time your things take, like allocation. You don’t have pause times. And it is a well-designed language. I think it is better than C++ which we are currently using. So if we were starting at this point in time, I would take a hard look at Rust, and I imagine that we would pick it instead of C++. Of course, when we started Rust didn’t have the maturity that it has now, but it has progressed a long time since then and I’m following it with great interest. I think it’s a well-done language.
Re: Scylla – Real-Time Big Data Database
#8The benchmarks against DynamoDB, Bigtable, & CockroachDB [1] appear quite impressive - anyone have real world experience that can attest to these claims of improved performance and reduced cost? > Scylla vs DynamoDB – Database Benchmark > 20x better throughput in the hot-partition test > Scylla Cloud is 1/7 the expense of DynamoDB when running equivalent workloads > Scylla Cloud: Average replication latency of 82ms.…
I was unwilling to sign up to read the actual benchmark report for the comparison to cockroachdb but it jumped out at me as odd. They solve completely different kinds of problems in my experience so I’m not surprised Scylla did better in raw throughout. That’s not interesting though. It would be just as weird for cockroach to put up a benchmark showing it outperforms in distributed sql queries.
That said I’ve seen the value Scylla brings in its core value prop, replacing Cassandra. It’s real good at that.
Re: Scylla – Real-Time Big Data Database
#9I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
If you are building a database engine that strongly prioritizes performance, and Scylla does position itself that way, then C++ is the only practical choice today for many people, depending on the details. It isn't that C++ is great, though modern versions are pretty nice, but that it wins by default.
Garbage collected languages like Golang and high-performance database kernels are incompatible because the GC interferes with core design elements of high-performance database kernels. In addition to a significant loss of performance, it introduces operational edge cases you don't have to deal with in non-GC languages.
Rust has an issue unique to Rust in the specific case of high-performance database kernels. The internals of high-performance databases are full of structures, behaviors, and safety semantics that Rust's safety checking infrastructure is not designed to reason about. Consequently, to use Rust in a way that produces equivalent performance requires marking most of the address space as "unsafe". And while you could do this, Rust is currently less expressive than modern C++ for this type of code anyway, so it isn't ergonomic either.
C++ is just exceptionally ergonomic for writing high-performance database kernels compared to the alternatives at the moment.
Re: Scylla – Real-Time Big Data Database
#10I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
If you are building a database engine that strongly prioritizes performance, and Scylla does position itself that way, then C++ is the only practical choice today for many people, depending on the details. It isn't that C++ is great, though modern versions are pretty nice, but that it wins by default. Garbage collected languages like Golang and high-performance database kernels are incompatible because the GC interfe…
At QuestDB we chose zero-GC Java for 80% of the code base, which resulted in superior performance on ingestion compared to the alternatives.