When are they going through with a rename?
This sub-optimal name is holding them back.
How We Built a Vectorized SQL Engine
11–20 of 56 posts
Re: How We Built a Vectorized SQL Engine
#12This is such a great post! Soooo many databases have implemented some kind of “column store extension” where they change the on disk representation and declare “look now we’re a hybrid database”. But the academic literature suggests that the bigger win for column stores may be in the execution layer. I love that Cockroach has taken the opposite of the usual approach and been so rigorous in measuring everything. After…
It's hard to say whether one would generally be a bigger gain than another without experimenting. We chose to focus on just execution because it was a relatively smaller project (with less side-effects) that still promised a large performance improvement. In the future, we might consider keeping data in columnar format on learner replicas to offer even better performance for users that would like to run OLAP-style queries on slightly stale data, but this would be a larger project.
Re: How We Built a Vectorized SQL Engine
#13Very nice post.
On another note: Kdb has figured out how to optimize the heck out of columnar data stores and vector processing. They would be my benchmark for these types of optimizations.
Re: How We Built a Vectorized SQL Engine
#14When are they going through with a rename? This sub-optimal name is holding them back.
Why is this sub-optimal? People may not like Cockroaches but they definitely have a reputation for durability, which isn't a bad mental link for a database company. It also provides an identity & personality which is so often missing from modern companies. Maybe they've found they get more from those who really like it than those who really hate it (or maybe they just like it themselves).
Re: How We Built a Vectorized SQL Engine
#15When are they going through with a rename? This sub-optimal name is holding them back.
Why is this sub-optimal? People may not like Cockroaches but they definitely have a reputation for durability, which isn't a bad mental link for a database company. It also provides an identity & personality which is so often missing from modern companies. Maybe they've found they get more from those who really like it than those who really hate it (or maybe they just like it themselves).
It still has a negative connotation. That's unavoidable.
People don't like cockroaches.
From a marketing point of view, it is a very bad name.
Unless the perception of cockroaches changes, which is rather unlikely.
Re: How We Built a Vectorized SQL Engine
#16This is such a great post! Soooo many databases have implemented some kind of “column store extension” where they change the on disk representation and declare “look now we’re a hybrid database”. But the academic literature suggests that the bigger win for column stores may be in the execution layer. I love that Cockroach has taken the opposite of the usual approach and been so rigorous in measuring everything. After…
This is a complicated discussion because the relative performance is greatly impacted by design and implementation details. Broadly speaking, the execution engine and page representation designs are tightly coupled for performance optimization purposes. The kinds of optimizations outlined at the link will have a high return for most kinds of databases.
The term "columnar" covers a diverse set of architectures with very different operational characteristics. For OLTP (like CockroachDB), using a classic DSM-style columnar representation is going to offer poor write performance no matter what you do with the execution engine. On the other hand, if you are using one of the newer vectorized page representations (VSM) that are popular for mixed workloads, which are quasi-columnar but not DSM (nor one of the intra-page DSM hybrids like PAX), the loss of write performance may be minimal versus a classic row store (NSM) but with much faster query processing (faster than DSM for some types of queries). The execution engine design you would attach to any of these models is pretty different.
Note also that a practical limit on this kind of optimization is code complexity. While VSM-style on-disk representation and matching execution engine sounds like a nearly optimal hybrid of both NSM for write performance and DSM for query performance, an implementation that is general purpose and performs well across a diverse set of data models is massively more difficult and complex to build in practice so most database designers avoid it at all costs due to the engineering overhead. These tend to be more common when the set of supported data models are very limited at design time. It is a research area that still offers a lot of opportunity -- the literature mostly ignores parts of the productive design space that intrinsically have extremely high implementation complexity (too difficult to produce code in support of paper publication).
Re: How We Built a Vectorized SQL Engine
#17When are they going through with a rename? This sub-optimal name is holding them back.
Why is this sub-optimal? People may not like Cockroaches but they definitely have a reputation for durability, which isn't a bad mental link for a database company. It also provides an identity & personality which is so often missing from modern companies. Maybe they've found they get more from those who really like it than those who really hate it (or maybe they just like it themselves).
Also, Dung Beetles Navigate Using the Milky Way.
Still wouldn't call my new browser Dung Beetle...
Re: How We Built a Vectorized SQL Engine
#18> A Datum now has a field for each possible type it may contain, rather than having separate interface implementations for each type. There is an additional enum field that serves as a type marker, so that when we do need to, we can inspect a type of a Datum without doing any expensive type assertions. This type uses extra memory due to having a field for each type, even though only one of them will be used at a time.
> The Go templating engine allows us to write a code template that, with a bit of work, we can trick our editor into treating as a regular Go file. We have to use the templating engine because the version of Go we are currently using does not have support for generic types.
Go’s lack of an expressive type system continues to disappoint :(
Re: How We Built a Vectorized SQL Engine
#19When are they going through with a rename? This sub-optimal name is holding them back.
You might be interested in BikeshedDB [0].
Re: How We Built a Vectorized SQL Engine
#20Earlier quoted context omitted.
Why is this sub-optimal? People may not like Cockroaches but they definitely have a reputation for durability, which isn't a bad mental link for a database company. It also provides an identity & personality which is so often missing from modern companies. Maybe they've found they get more from those who really like it than those who really hate it (or maybe they just like it themselves).
It still has a negative connotation. That's unavoidable. People don't like cockroaches. From a marketing point of view, it is a very bad name. Unless the perception of cockroaches changes, which is rather unlikely.
If you pick a happy name, you get complaints that they sound too generic, sound like a medication name, etc. People also complain if your name ends in .com or .net, as apparently that sounds like something from the mid 90s.
People just like complaining about names.