Live data from Hacker News

Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

databeta.wordpress.com

21–30 of 77 posts

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#21
Being faster (in throughput) than Redis doesn't seem that difficult, Redis is mostly single core. But with that restriction comes the magical constraint that you can run Lua on that single core without having to think about race conditions or concurrency. For many complicated systems this is an incredible property and can provide incredible power. Yes, you may eventually outgrow Redis, but if you do then you are entering pretty crazy territory where you will likely need something custom anyways.

All that to say, that particular comparison feels a bit Apples and Oranges to me.

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#22
post #5

I feel like between Redis, S3, Cloud Storage, RocksDB, Cassandra, etc...this area strikes me as one that has been solved as well as we could reasonably expect it to be. What the world of data needs more of is continued development into novel indexing strategies/implementations. ElasticSearch, Postgres's GIN index on JSONB, MapReduce, graph databases. I don't need another key value store...

Cassandra is JVM GC issues Redis is in-memory only Cloud Storage - Not sure, how we can use it outside of cloud vendors RocksDB - Facebook just outsourced the engine to the community, where is the service which adds replication, clustering and network interface on top of it? I am sure, they use one internally, why is it not being open-sourced? There is also badger but most of these only offer low-level operation. Sor…

> where is the service which adds replication, clustering and network interface on top of it?

Actually I would prefer such kind of database (just the engine).

1. Expose it to network via what ever framework you like. Thrift, Rest, grpc, ... You don't have to include different kind of libraries for each network service. I would love to connect to every network service (Redis, Elasticsearch, Cassandra, MySQL, ...) via a single framework (say grpc).

2. In most large scale scenarios, there is already some kind of log service (DistributedLog, NATS, Kafka, ...). Why not take benefit of that for replication? Isn't it great to separate the engine layer from replication layer? Currently we are doing double replication actually. Replicate data from master DB to slave DB. Then replicate the same data, from any DB to cache, search, ... components. The data is already there on log. Let everyone (slave DB as well as cache/search module) consume it. This is basically state machine replication idiom. PNUTS[0], Twitter K/V database, LinkedIn Espresso [1] (as well as Ambry[2] which is their internal object store), ... use this approach for replication.

3. I would agree with that, they only support basic low level operations.

[0] http://www.vldb.org/pvldb/1/1454167.pdf

[1] http://www.csce.uark.edu/~xintaowu/BDAM/p1135-qiao.pdf

[2] http://dprg.cs.uiuc.edu/docs/SIGMOD2016-a/ambry.pdf

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#23
post #5

I feel like between Redis, S3, Cloud Storage, RocksDB, Cassandra, etc...this area strikes me as one that has been solved as well as we could reasonably expect it to be. What the world of data needs more of is continued development into novel indexing strategies/implementations. ElasticSearch, Postgres's GIN index on JSONB, MapReduce, graph databases. I don't need another key value store...

Cassandra is JVM GC issues Redis is in-memory only Cloud Storage - Not sure, how we can use it outside of cloud vendors RocksDB - Facebook just outsourced the engine to the community, where is the service which adds replication, clustering and network interface on top of it? I am sure, they use one internally, why is it not being open-sourced? There is also badger but most of these only offer low-level operation. Sor…

Cassandra GC: check out scylla or the rocksandra projects.

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#24
"In the style of existing systems such as Cassandra and Bayou, programmers can embed application-specific conflict resolution logic into the merge function of a Anna ValueLattice. Anna gives the programmer the freedom to program their ValueLattices in this ad hoc style, and in these cases guarantees only replica convergence. We define this level of ad hoc consistency as simple eventual consistency."

So... is that kind of like Riak where (I believe Riak would do this) would send all conflicting values to the client for resolution, but instead you provide resolution strategies as part of the query? My understanding of vector clocks is admittedly pretty shitty though.

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#28
This link claims to be 700x faster than something called Masstree, which itself claims to do about 3 million requests per second with 16 cores. I'm not sure I buy 131 Million request per second per core.

> it was up to 700x faster than Masstree, up to 800x Intel’s “lock-free” TBB hash table. In fairness, those systems provide linearizable consistency and Anna does not. But Anna was still up to 126x faster than a “hogwild”-style completely inconsistent C++ hashtable due to cache locality for private state, while providing quite attractive coordination-free consistency.

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#29

Can Anna's approach improve current solutions to the problem of managing secondary indexes in a partitioned KV store while preserving consistency?

I don't think this problem exists outside of the realms of linearizable or serializable consistency, which this system doesn't provide.

Re: Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store

#30
post #26

Rant: Why so much focus on Key Value stores? That's the easy part of the problem. I would like to know more about the interesting ones: secondary indexes, range scans, performance on mixed workloads, robustness, operational complexity.

R-trees (and their higher-dimensional counterparts) and B/B+-trees are great for supporting range queries. You’re right that kvs systems are limited, and I find the above quite effective at supporting more general operations.
Post reply on HN