Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building CockroachDB on top of RocksDB

cockroachlabs.com

Post reply on HN
Print view View original

21–30 of 79 posts

Re: Building CockroachDB on top of RocksDB

#21

Post by ryanworl »

perfmode wrote: post #19
ryanworl wrote: post #16

Earlier quoted context omitted.

The CockroachDB blog post on this topic is a good summary. There is an additional trick that isn't directly KV related, but is important in a distributed environment when using a KV storage engine. When defining a hierarchy of tables, such as customers -> orders -> order_line_items, you can make the primary key of the child tables contain the primary key of the parent table. e.g. (customer_id) for the customers table…

Can you link to the post you’re referring to?

https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mappin... this was in another comment.

https://emsal.me/blog/5 this blog post has a good introduction to their implementation of interleaved tables.

Re: Building CockroachDB on top of RocksDB

#24

Post by ddorian43 »

ryanworl wrote: post #20
gigatexal wrote: post #17

Earlier quoted context omitted.

Interleaved tables are best for 1:1 relationships.

Hard disagree. This is the only way in a distributed, sorted KV store to get any semblance of data locality. If Cockroach and Spanner didn't do this, they would constantly be doing 2PC for modifying data that is related but stored on different groups of machines.

You can have hash sharding and be able to distribute tables easily compared to range sharding.

Re: Building CockroachDB on top of RocksDB

#25

Post by gianm »

polskibus wrote: post #6
ryanworl wrote: post #5

Earlier quoted context omitted.

For analytics workloads, your best bet is using compression techniques that let you do operations on the data without decompressing it. A good example is dictionary encoding a set of sorted string keys so you can preform prefix queries by doing a greater than and less than comparison on the integers instead of examining every string entirely. Once you’ve encoded the data into large enough blocks, you could use any st…

I know there are many techniques that used together give good performance (optimal memory layout, compression, vectorization, etc. etc.), however I'd like to use a package that does a lot of it, same what RocksDB (or SQLite) does for OLTP cases. Is there something like that? If not, what's out there that gives the best foundation for building OLAP functionalities on top of it?

Check out Druid [1], an open-source analytical database with tightly-coupled storage and processing engines designed for OLAP. In particular it implements a memory-mappable storage format, indexes, compression, late tuple materialization, and query engines that can operate directly on compressed data. There is a patch out to add vectorized processing as well, so you should expect to see that show up in a future release.

Its storage format and processing engine aren't designed to be embedded in the same way as RocksDB and SQLite are, but you certainly could if you wanted to, since the code is fairly modular. Or you could use it as a standalone service as it was designed to be used.

[1] http://druid.io/

Re: Building CockroachDB on top of RocksDB

#26

Post by the_duke »

Excellent article, very informative.

I just had to chuckle at this:

> Non-engineers: in a computer, a move is always implemented as a copy followed by a delete

Yeah, that's really gonna help a non-developer understand the article better...

Re: Building CockroachDB on top of RocksDB

#28

Post by georgewfraser »

arjunnarayan wrote: post #11
ryanworl wrote: post #10

Earlier quoted context omitted.

No, I am not aware of any storage engine that provides that out of the box. The techniques are very tied into what your query processing engine can do and expects the data to look like. For example, do you materialize tuples immediately, or do you fully run it through your processing pipeline and not materialize until the end? Your storage engine and format needs to be at least somewhat involved in answer that questi…

Unfortunately most of the systems that build what you're describing are closed source (e.g. Snowflake, Microsoft SQL Server, Vertica, Teradata). There isn't an open-source project that does all of those things.

What about Presto?

Re: Building CockroachDB on top of RocksDB

#29

Post by tyingq »

the_duke wrote: post #26

Excellent article, very informative. I just had to chuckle at this: > Non-engineers: in a computer, a move is always implemented as a copy followed by a delete Yeah, that's really gonna help a non-developer understand the article better...

It's confusing altogether. For example, that's not how /bin/mv (usually) works.

Re: Building CockroachDB on top of RocksDB

#30

Post by jandrewrogers »

polskibus wrote: post #2

I noticed that RocksDB is used very often in OLTP scenarios. What's the OLAP equivalent of RocksDB in OLTP world? Apache Parquet? Apache Arrow? What would you use these days to create a high performance OLAP/OLHybridP engine ?

There is a practical engineering reason why the OLAP equivalent doesn't seem to exist. General purpose storage engines, and this applies to RocksDB, are like the C++ STL in that they provide good average performance across a wide range of common cases but are nowhere close to optimal if you have a well-defined type of data model and workload as your use case. You can always gain an integer factor increase in throughput by designing a less generalist implementation with a similar interface.

As with the C++ STL, the limiting factor is the number of tunable parameters available i.e. the amount of internal architectural flexibility built into the implementation. OLTP storage engines are pretty simple, so a manageable number of behavioral parameters can usually get you within 3x of the throughput of a more targeted design, which is acceptable performance for most workloads that are not ingest-intensive.

OLAP-ish storage engines, on the other hand, are at least an order of magnitude more complex to implement and have many more degrees of freedom depending on the expected data model and workload. There is a lot more data model and workload diversity in OLAP than OLTP, which makes implementing the effective internal architectural flexibility and set of tunable parameters that need to be maintained very unwieldy. If you limited yourself to the number of user-definable tuning and configuration parameters as an OLTP-oriented storage engine like RocksDB, the performance gap between a generalist implementation and a more targeted implementation will be more like 10-100x, which needless to say is huge. This makes the practical applicability of any "general purpose" OLAP storage engine that someone would want to use quite narrow, which diminishes the value of implementing a general purpose engine.

This leads to the current reality that there is a zoo of specialist storage engines for OLAP-ish workloads -- graph, time-series, event processing, geospatial, classic DW, etc. Much more generalist OLAP storage engines that do several of these models could exist in theory but the bar for technical sophistication and complexity is much higher than for OLTP.

Open source projects in particular tend to have a natural ceiling on the number of man-years invested to get an initial implementation of an architecture, which inherently limits the expressiveness of that architecture for software with this complexity.

Post reply on HN