Columnar Storage
11–18 of 18 posts
Re: Columnar Storage
#12possibly dumb question; what's the difference between columnar storage and a bunch of row-major tables with an id column, and one column of data?
In database systems it is important to distinguish between the logical and physical models.
When you design a relational database, you focus on the correct logical model.
"People can have multiple phone numbers".
"Phone numbers belong to a single phone".
"Mobile phones are possessed by one person. Landlines can be shared".
And so on. You express this logical model to the database, most likely in SQL.
Eventually you notice that query X is slow. Your first step is to check that your logical design was sound, because poorly designed schemata are hard for query planners to reason correctly about.
Then you start doing things to the physical representation. You say stuff like:
"I look up by phone numbers a lot."
Or, in SQL terms, you add an index to a column.
Similarly, as this article pointed out, there are times when grouping data by column rather than row is advantageous. So then you tell the database to use a columnar store.
And so on. Modern RDBMSes all support the same major logical model descriptions; but they can vary widely on what physical directives you give.
Re: Columnar Storage
#13possibly dumb question; what's the difference between columnar storage and a bunch of row-major tables with an id column, and one column of data?
See the paper "Column-stores vs. row-stores: how different are they really?" in SIGMOD 2008 for performance comparisons between C-store and approaches of emulating column-store in row-store databases.
Re: Columnar Storage
#14Earlier quoted context omitted.
Sounds like Impala could use your talent. You should talk with them.
Thanks for the pointer! Looking at Cloudera's jobs board, most tech-heavy work is in SF or Palo Alto. I live in San Mateo, convenient to both of those, but ... I'm likely leaving for the North Carolina RTP area soon.
Re: Columnar Storage
#15Re: Columnar Storage
#16It's good to see this is finally catching on not only for BI and datamining tasks.
Re: Columnar Storage
#17This is an in-memory column+row store which handles both OLAP and OLTP workloads neatly