31–40 of 79 posts
https://stackshare.io/stream/stream-and-go-news-feeds-for-ov...
Excellent article, very informative. I just had to chuckle at this: > Non-engineers: in a computer, a move is always implemented as a copy followed by a delete Yeah, that's really gonna help a non-developer understand the article better...
It's confusing altogether. For example, that's not how /bin/mv (usually) works.
Earlier quoted context omitted.
I know there are many techniques that used together give good performance (optimal memory layout, compression, vectorization, etc. etc.), however I'd like to use a package that does a lot of it, same what RocksDB (or SQLite) does for OLTP cases. Is there something like that? If not, what's out there that gives the best foundation for building OLAP functionalities on top of it?
Check out Druid [1], an open-source analytical database with tightly-coupled storage and processing engines designed for OLAP. In particular it implements a memory-mappable storage format, indexes, compression, late tuple materialization, and query engines that can operate directly on compressed data. There is a patch out to add vectorized processing as well, so you should expect to see that show up in a future relea…
Disclosure: I work at Percona, and we've seen a lot of our customers make use of Clickhouse and have begun some of our own services work around it in Consulting. It's now a primary database talked about at our conferences, and we post about it regularly. [3]
[1]: https://clickhouse.yandex/ [2]: https://blog.cloudflare.com/how-cloudflare-analyzes-1m-dns-q... [3]: https://www.percona.com/blog/2018/10/01/clickhouse-two-years...
Earlier quoted context omitted.
Check out Druid [1], an open-source analytical database with tightly-coupled storage and processing engines designed for OLAP. In particular it implements a memory-mappable storage format, indexes, compression, late tuple materialization, and query engines that can operate directly on compressed data. There is a patch out to add vectorized processing as well, so you should expect to see that show up in a future relea…
There's also Clickhouse [1] which seems to scale much better than Druid, and has similar architectural decisions to make it somewhat general as a columnar store for OLAP uses. Cloudflare wrote an article in the past where they compared Clickhouse and Druid and they chose Clickhouse because they could get similar performance on the same workload with 9 nodes in Clickhouse which would require hundreds for Druid. They b…
Is this actually true?
spark, foundationdb, memsql, nuodb , citus . I am not sure any of these are built on top of rocksdb.
Which ones are actually built on lsm?
> If you surveyed most NewSQL databases today, most of them are built on top of an LSM, namely, RocksDB. Is this actually true? spark, foundationdb, memsql, nuodb , citus . I am not sure any of these are built on top of rocksdb. Which ones are actually built on lsm?
Earlier quoted context omitted.
It's confusing altogether. For example, that's not how /bin/mv (usually) works.
Usually, that's because /bin/mv is just changing a link to the file, not moving the file itself. In cases where it's actually moving the file -- say across a file system boundary -- it does copy the file and then delete the old version.
Our in-house DB at Stream also runs on top of RocksDB + Raft. Its amazing just how much faster it is than anything else out there (especially compared to cassandra). Instagram uses rocksdb as storage for Cassandra, Linkedin and pinterest use rocksdb. As soon as you have the time to build your own db using rocksdb you get really finegrained control over performance. https://stackshare.io/stream/stream-and-go-news-feed…
Our in-house DB at Stream also runs on top of RocksDB + Raft. Its amazing just how much faster it is than anything else out there (especially compared to cassandra). Instagram uses rocksdb as storage for Cassandra, Linkedin and pinterest use rocksdb. As soon as you have the time to build your own db using rocksdb you get really finegrained control over performance. https://stackshare.io/stream/stream-and-go-news-feed…
Rocksdb is pretty good and we relied heavily on it at QuasarDB as well. Having said that, we are nowadays deploying more and more production setups with Levyx’ Helium, which scales better and directly integrates with the hardware.
Earlier quoted context omitted.
Check out Druid [1], an open-source analytical database with tightly-coupled storage and processing engines designed for OLAP. In particular it implements a memory-mappable storage format, indexes, compression, late tuple materialization, and query engines that can operate directly on compressed data. There is a patch out to add vectorized processing as well, so you should expect to see that show up in a future relea…
There's also Clickhouse [1] which seems to scale much better than Druid, and has similar architectural decisions to make it somewhat general as a columnar store for OLAP uses. Cloudflare wrote an article in the past where they compared Clickhouse and Druid and they chose Clickhouse because they could get similar performance on the same workload with 9 nodes in Clickhouse which would require hundreds for Druid. They b…
[1]:https://medium.com/@leventov/comparison-of-the-open-source-o...