What's some concrete example of OLAP cube? What does Alice, a data analyst, actually do when she gets to work at her computer? What does she interact with on the screen? Does she use some specialized software to project the data cube into two dimensions (contingency tables) to find hidden meaning in the data? There's a lot of abstract talk and no actual examples on the Internet, except for SQL Server tutorials which always end up with some kind of E-R diagram.
The Rise and Fall of the OLAP Cube
41–50 of 54 posts
Re: The Rise and Fall of the OLAP Cube
#42For anyone wondering, OLAP != OLAP Cubes OLAP = A category of databases meant for analyzing data. These are eventually consistent db's, and not OLTP db's. OLAP db's include Redshift, Teradata, Snowflake, BigQuery, and others. Generally what makes a database an MPP database is partitioning compute and storage. Generally what differentiates one MPP db from another is whether or not data and compute are colocated. OLAP…
Not unique to MS SQL Server - Oracle and IBM (InfoSphere) have cube engines, and there have been independent implementations as well: https://en.wikipedia.org/wiki/Comparison_of_OLAP_servers
That’s a valid point. Thanks.
Re: The Rise and Fall of the OLAP Cube
#43Thanks for sharing. Interesting write up. While this article accurately captures the issues with traditional OLAP Cubes, it failed to recognize the latest development in this domain. Projects like Apache Kylin, and its commercial version Kyligence, leverage modern computer architectures such as columnar storage, distributed processing, and AI optimization to build cubes over 100s of billions rows of data that covers…
> And, comparing columnar stores to OLAP cubes is like comparing apples to oranges. The former is a storage format and the latter is an analytical pattern. Modern OLAP cube technology like Kylin/Kyligence stores cubes in columnar stores anyway.
This is mistaken. I went back to read most of the academic literature on OLAP cubes while working on this piece (which, unlike vendor marketing, is used with consistency since the early 80s). OLAP cubes or data cubes refer specifically to the data structure that grew out of nested arrays. An OLAP cube may be materialized from a column store, but a column store isn't an OLAP cube.
Relevant sources are included at the bottom of the piece.
Re: The Rise and Fall of the OLAP Cube
#44well, I would like to say the OLAP Cube is just re-rising now. There are 1000+ companies deployed Apache Kylin (OLAP Engine for Big Data) in the past 5 years, for 100+B rows, for 100+ concurrent users...many different use cases are based that technology...it works very well with BI tools and so friendly to analysts who are using such "old fashion" every day over the decade (how hard for them to be Data Scientists?) c…
+1 for Apache Kylin, it's a great project and awesome open source community. If anyone is curious about what modern OLAP is capable of, check it out. If you want an alternative take on where OLAP is today and what it is capable of, I usually recommend this article: https://kyligence.io/blog/olap-analytics-is-dead-really/
There are huge problems with that piece. For starters, OLAP != OLAP cube. Columnar databases and OLAP cubes are both designed for OLAP workloads. They are simply different architectures. Therefore, it is impossible to argue that 'OLAP is dead' — it cannot be dead, because OLAP is simply a type of database usage.
At this point you might say, "oh, OLAP cubes refer to an abstraction, it can be implemented using columnar stores!" — and I would point you to 40 years worth of academic research that stretches back to the early 80s. The OLAP cube or data cube refers to a specific type of data structure. It just so happens that vendors like to use the term 'OLAP cube' even when they are using a columnar engine under the hood, because it sells well.
Re: The Rise and Fall of the OLAP Cube
#45It seems that the article makes a categorical error, arguing that OLAP cubes were replaced by columnar data stores. I always understood OLAP cube as an abstract concept that can have various technical implementations, while column store is a kind of optimization in that technical implementation.
I went back to read most of the academic literature on OLAP cubes while working on this piece (which, unlike vendor marketing, is used with consistency since the early 80s). OLAP cubes or data cubes refer specifically to a data structure. An OLAP cube may be materialized from a column store, but a column store isn't an OLAP cube.
The proof of this? Go to any serious columnar database provider and search for the words 'OLAP cube'. You will find that they are careful to say 'OLAP workload', but not 'OLAP cube' — because in the strict definition of the term, an OLAP cube or data cube is an entirely different architecture.
Relevant sources are included at the bottom of the piece.
endorphone's comment has it right.
Re: The Rise and Fall of the OLAP Cube
#46Two false statements in this article: > ...Amazon, Airbnb, Uber and Google have rejected the data cube... Airbnb uses Druid which is essentially an OLAP cube. > BigQuery, for instance, doesn’t allow you to update data at all It's not like that anymore since several years.
Amazon uses Apache Kylin, which is a modern OLAP cube technology.
Amazon also uses Redshift, which is a modern columnar database.
An example does not an argument make.
Re: The Rise and Fall of the OLAP Cube
#47For anyone wondering, OLAP != OLAP Cubes OLAP = A category of databases meant for analyzing data. These are eventually consistent db's, and not OLTP db's. OLAP db's include Redshift, Teradata, Snowflake, BigQuery, and others. Generally what makes a database an MPP database is partitioning compute and storage. Generally what differentiates one MPP db from another is whether or not data and compute are colocated. OLAP…
Thank you for saying this! I think your definition of OLAP is good, but the definition of OLAP cube could simply be 'a data structure that is basically a very sophisticated nested array'. I'm oversimplifying, of course — there are over four decades of work at this point on OLAP cubes, including various compression schemes (e.g. https://link.springer.com/chapter/10.1007/978-3-642-17569-5_...) and other techniques to aggregate data when the cube outstrips the hosts's memory. But by and large it is important to understand that an OLAP cube is simply a data structure ... and different vendors have different implementations and optimisations of said data structure.
I think this confusion exists because too many vendors conflate the two terms. They say OLAP when they mean OLAP cube. BigQuery and Redshift, however, do not: they are very clear that their dbs are designed for OLAP workloads but are not cubes.
Re: The Rise and Fall of the OLAP Cube
#48For anyone wondering, OLAP != OLAP Cubes OLAP = A category of databases meant for analyzing data. These are eventually consistent db's, and not OLTP db's. OLAP db's include Redshift, Teradata, Snowflake, BigQuery, and others. Generally what makes a database an MPP database is partitioning compute and storage. Generally what differentiates one MPP db from another is whether or not data and compute are colocated. OLAP…
Redshift is not eventually consistent, it's fully ACID compliant.
Re: The Rise and Fall of the OLAP Cube
#49Earlier quoted context omitted.
Your users want their dashboards and reports to contain data that's as up-to-date as possible. No one is happy about waiting 24 hours or more for a batch update to run if there are decisions they need to make right now. The difference vs. OLTP is that it might be acceptable for events to take minutes or even hours to be reflected in the database, vs. ~seconds or less for a transaction where the user is interacting an…
Most of the users I work with, when asked why they want more than daily batches, respond with something emotional and not business value. I had a request for real-time updates for a massive metric that is based around monthly usage. Ya just gotta push back, and hep people understand what the trade offs are for real-time. Most people don’t need it.
Absolutely true. The ones that do, do. For example, multiuser planning systems absolutely do. And those were what OLAP was originally intended to support.
Re: The Rise and Fall of the OLAP Cube
#50This piece reiterates a bunch of the usual misunderstandings of OLAP. Likely because it’s following the DBMS community’s redefinition of OLAP towards cross-tab group-by, and away from the higher end of the OLAP market. The “cube” is just as much a logical construct as a SQL table is, and just because row stores were bad at certain analytics didn’t mean that SQL was. Compressed column-stores hurt OLAP, because update…
> Compressed column-stores hurt OLAP, because update throughput (the “on-line” in OLAP) is relatively bad. Uncompressed / array stores are quite good. But why do you want high update throughout when you're doing mostly reads? Every definition I read about OLAP says this is one of the fundamental differences, or am I misunderstanding something?
You’re missing the important definitions from back when OLAP originated. There is a high update rate for the use cases I build software for.