Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rise and Fall of the OLAP Cube

holistics.io

Post reply on HN
Print view View original

11–20 of 54 posts

Re: The Rise and Fall of the OLAP Cube

#11

Post by quantified »

This piece reiterates a bunch of the usual misunderstandings of OLAP. Likely because it’s following the DBMS community’s redefinition of OLAP towards cross-tab group-by, and away from the higher end of the OLAP market. The “cube” is just as much a logical construct as a SQL table is, and just because row stores were bad at certain analytics didn’t mean that SQL was.

Compressed column-stores hurt OLAP, because update throughput (the “on-line” in OLAP) is relatively bad. Uncompressed / array stores are quite good.

The quantity and complexity of SQL necessary for a non-trivial OLAP view is daunting: time-series views (mix YTD and current-period calcs for transactional and balance accounts, get the ratio measures calculated, and handle the joins necessary for calculating and aggregating from the many (easily dozens) of fact tables that have different dimensionality and granularity and need to be joined at their finest detail. The user will want to see a set of metrics for a pair of orgsnizational units, and also as a percentage difference between the two org units. SQL does not have inter-row calculations, so quite a bit of work goes into re-shaping the SQL cursor’s results to something the user wants to see.

All that query generation and result transformation is part of the value-add of the “cube” server.

So the OLAP cube as a logical construct is definitely not fallen. Just the bad ones. They’re less flexible than SQL but provide way more productivity within the query space they’re built for.

Re: The Rise and Fall of the OLAP Cube

#12

Post by lou1306 »

> Codd got called out for his conflict of interest and was forced to retract his paper … but without much fallout, it seems: today, Codd is still regarded as ‘the father of the relational database’

I found this passage confusing. He is regarded as such because of his work on the relational algebra, and the shady OLAP backstory is unrelated to that.

Re: The Rise and Fall of the OLAP Cube

#13

Post by simo7 »

Two false statements in this article:

> ...Amazon, Airbnb, Uber and Google have rejected the data cube...

Airbnb uses Druid which is essentially an OLAP cube.

> BigQuery, for instance, doesn’t allow you to update data at all

It's not like that anymore since several years.

Re: The Rise and Fall of the OLAP Cube

#14

Post by simo7 »

I think the article is a bit simplistic.

It's true that _often_ OLAP cubes are not needed. That's simply because the amount of data and the latency requirements are _often_ not too demanding.

Also, materialized views don't solve the major issue with OLAP cubes: the need of maintaining data pipelines.

I wonder if a solution to this problem could come from a different way of caching result sets: new queries that would produce a subset of a previously cached result could be run against the cached result itself. Of course this opens up a new set of problems, cache invalidation etc..

Re: The Rise and Fall of the OLAP Cube

#15

Post by tomrod »

_Codemonkeyism wrote: post #3

After maintaining an OLAP cube system for some years, I'm not that sure after reading the article. The nice thing of an OLAP cube is the UI and how business users can easily drag and drop items to explore data (standard reports are best created automatically and don't need an OLAP layout/setup). If the UI (Tableau, Excel Power Pivot) is the same, then yes, OLAP cubes are a thing of the past. Otherwise not.

It's basically the same.

Re: The Rise and Fall of the OLAP Cube

#16

Post by js8 »

It seems that the article makes a categorical error, arguing that OLAP cubes were replaced by columnar data stores. I always understood OLAP cube as an abstract concept that can have various technical implementations, while column store is a kind of optimization in that technical implementation.

Re: The Rise and Fall of the OLAP Cube

#17

Post by lukehan »

well, I would like to say the OLAP Cube is just re-rising now. There are 1000+ companies deployed Apache Kylin (OLAP Engine for Big Data) in the past 5 years, for 100+B rows, for 100+ concurrent users...many different use cases are based that technology...it works very well with BI tools and so friendly to analysts who are using such "old fashion" every day over the decade (how hard for them to be Data Scientists?) check more here: http://kylin.apache.org/community/poweredby.html

Re: The Rise and Fall of the OLAP Cube

#18

Post by lukehan »

js8 wrote: post #16

It seems that the article makes a categorical error, arguing that OLAP cubes were replaced by columnar data stores. I always understood OLAP cube as an abstract concept that can have various technical implementations, while column store is a kind of optimization in that technical implementation.

that's true

Re: The Rise and Fall of the OLAP Cube

#19

Post by greggyb »

arethuza wrote: post #10
sgt wrote: post #7

How does something like Tableau fit in? I know of people using Tableau with a Postgres connector, but I am not sure if that allows you the same kind of performance as you'd get with OLAP or even a columnar DB.

I don't know about Tableau, but Microsoft Power BI does use a columnar engine - I believe its the same VertiPaq/xVelocity component used in Excel Power Pivot and SSAS Tabular Models.

Correct. It also shares a fair bit of implementation with columnstore indices in MSSQL Server.

There's also a proprietary relational query language shared among Power BI, Power Pivot, and SSAS Tabular, called DAX.

Re: The Rise and Fall of the OLAP Cube

#20

Post by endorphone »

js8 wrote: post #16

It seems that the article makes a categorical error, arguing that OLAP cubes were replaced by columnar data stores. I always understood OLAP cube as an abstract concept that can have various technical implementations, while column store is a kind of optimization in that technical implementation.

The article is generalizing, as such articles do. Necessarily so. There are always exceptions. It is unfortunate that when such articles appear the dominant response is nitpicking or exception pointing (I'd say 80%+ of the comments thus far).

In the overwhelming bulk of cases firms have a column store that they generate cubes from. OLAP is run against the cubes. Some put warehousing in between, though that changes little. Cubes are fundamentally a form of caching because historically it was prohibitive to do large-scale aggregations in real-time. With massive memory servers, and more importantly flash storage that improves aggregate performance at the enterprise scale by many magnitudes -- a million times faster analysis and aggregation is entirely possible -- that historic caching step becomes a hindrance and maintenance/timeliness issue. So it's discarded.

That's all 100% true. It has happened in many orgs.

Not all, of course. But as a general trend.

Post reply on HN