Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cube.js: Headless Semantic Layer

github.com

Post reply on HN
Print view View original

21–30 of 49 posts

Re: Cube.js: Headless Semantic Layer

#21

Post by totalhack »

pavel_tiunov wrote: post #17
totalhack wrote: post #4

Does cube support dynamically figuring out joins or is each cube a hard coded set of joins? My similar, much less polished project doesn't require you to specify joins ahead of time outside of optionally defining a tree-like table lineage: https://github.com/totalhack/zillion

Yep. Cube would automatically figure out the join path for you on top of the defined join graph in the data model using the Dijkstra algorithm. The best practice however is to use views: https://cube.dev/docs/schema/reference/view/#views . Those can be used to explicitly control join paths and get an effect similar to what Looker Explore can provide.

Cool, do you have an example showing a config/setup that can do dynamic joins? All the examples I see have joins explicitly defined at the cube level in the yaml config.

Zillion uses networkx to create a graph of tables and relationships so it sounds like it's doing something similar.

Views are a nice guardrail but can also get in the way! I've experienced this frustration first hand (and seen it with my business users) when using tools that focus on such a premise. Guess it depends how complex your data model is and how quickly your business is evolving / adding to that model. In the move-fast-break-stuff phase of a company they can get annoying at least.

Re: Cube.js: Headless Semantic Layer

#22

Post by rektide »

adolph wrote: post #10
rektide wrote: post #2

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. https://carlineng.com/?postid=semantic-layer#blog https://news.ycombinator.com/item?id=35715410 They indeed mentioned Cube.js. Bunch of Malloy links over time: https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDa... And cube.js, https://hn.algolia.com/?dat…

My operating theory is that in app centric development the data store is a component of the application. As a result, much of the metadata that changes data into information (data with meaning) is stored in app code and config. Upon ETL to reporting, warehouse, lake, or the like, the app’s semantic layer is lost.

It's a crying shame how little of the data-core makes it's way out of applications. For both the app's owning entity & especially for the users.

I paid respects[1] recently yet again to Window Manager Improved Improved (wmii), which kept it's state in a 9p filesystem any user could easily browse & modify. The state is still a component of the app/window manager, but it's at least malleable to all.

[1] https://news.ycombinator.com/item?id=35768686

Re: Cube.js: Headless Semantic Layer

#23

Post by sails »

pastacacioepepe wrote: post #6
rendall wrote: post #3

It looks very important, popular and well established, but what is it? I looked at the README hoping to understand what a semantic layer for building data applications means but no love. It helps data engineers and application developers access data from modern data stores, organize it into consistent definitions, and deliver it to every application. Like an ORM? Or a middleware? > Cube was designed to work with all…

Say you want to build a dashboard with charts and custom timerange selection using data you already have in Postgres/other DB, without killing your DB under the pressure of queries AND without having to write an additional API? Cube.js is the tool for that. Handles data modeling (you can define a schema on top of your SQL schema), caching, access control and API for you.

Data modelling is important to highlight, and if OP not familiar with the concept and need then likely won’t see the obvious value of Cube.

Re: Cube.js: Headless Semantic Layer

#24

Post by bbickell »

jspash wrote: post #14

There used to be a product called Statsbot which was a friendly UI built on top of Cube.js. They shuttered the service a few years back and we're still struggling to find an alternative that is as simple to use by anyone in the company and easy to set up. We've gone through all the usual BI suspects and nothing comes close. I'd love to find the time to rebuild this.

You might take a look at Delphi (https://www.delphihq.com/) which is built on top of semantic layers and recently integrated with Cube.

Re: Cube.js: Headless Semantic Layer

#26

Post by igorlukanin »

jspash wrote: post #14

There used to be a product called Statsbot which was a friendly UI built on top of Cube.js. They shuttered the service a few years back and we're still struggling to find an alternative that is as simple to use by anyone in the company and easy to set up. We've gone through all the usual BI suspects and nothing comes close. I'd love to find the time to rebuild this.

Oh, it's interesting to meet a Statsbot user in the wild! Indeed, Cube was spun off Statsbot and became the foundation on which others can build products like the one mentioned in the sibling comment: Delphi.

Cube acts as the semantic layer, providing the access to data sources and centralizing the data model. Delphi acts as the UI for the end user, enabling them to ask questions in natural language. I've blogged about Cube and Delphi here: https://cube.dev/blog/conversational-interface-for-semantic-.... Also, here's a demo video on YouTube I've recorded recently: https://www.youtube.com/watch?v=FotEaaf20gY

Re: Cube.js: Headless Semantic Layer

#27

Post by igorlukanin »

rektide wrote: post #2

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. https://carlineng.com/?postid=semantic-layer#blog https://news.ycombinator.com/item?id=35715410 They indeed mentioned Cube.js. Bunch of Malloy links over time: https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDa... And cube.js, https://hn.algolia.com/?dat…

So good you brought Malloy here. I like it quite a bit because the folks really try to innovate (heck, they even have their own data querying syntax to replace SQL). But what I like even more — being part of the Cube team — that the "cons" of existing solutions that Carlin mentions in his blog are actually already solved by Cube.

With Cube, Data exploration, ideation on the data model, querying, and bringing the insights all the way down to BI tools or data apps takes minutes rather than hours or days. Done, case closed :-)

Re: Cube.js: Headless Semantic Layer

#28

Post by igorlukanin »

rendall wrote: post #3

It looks very important, popular and well established, but what is it? I looked at the README hoping to understand what a semantic layer for building data applications means but no love. It helps data engineers and application developers access data from modern data stores, organize it into consistent definitions, and deliver it to every application. Like an ORM? Or a middleware? > Cube was designed to work with all…

As part of the Cube team, I have to admit that all descriptions in the sibling comments make a lot of sense. Of course, the "semantic layer" thing is quite known to data engineers/analysts and other data folks in general (they also know things like "metrics store", "headless BI", etc.) but not that well known outside of the data space. Probably, it would be best to describe what are the major use cases Cube is created for.

1. Embedded analytics — you have your data somewhere (data warehouse, database, etc.) and you'd like to embed it into a data app. Cube would provide connectivity to data sources, data modeling to define the metrics, caching to make your analytics fast, and APIs and SDKs to deliver them to the data app. E.g., if you decided to add a chart to your front-end app, fetching the data from the API would be as easy as sending a JSON query to Cube.

2. Semantic layer for the internal BI — you have your data somewhere and you'd like to provide access to insights based on that data to business users. Cube would provide connectivity to data sources, data modeling to define the metrics, access control to make sure only ones who need access to metrics have it, caching to make sure every dashboard loads instantly, and APIs to deliver the data to BI tools, notebooks, etc. E.g., if you want to create some dashboards in Superset, Metabase, Tableau, or Power BI, you'd just need to connect Cube's SQL API as if it was a regular database and start creating charts/dashboards.

Re: Cube.js: Headless Semantic Layer

#29

Post by igorlukanin »

igorlukanin wrote: post #28
rendall wrote: post #3

It looks very important, popular and well established, but what is it? I looked at the README hoping to understand what a semantic layer for building data applications means but no love. It helps data engineers and application developers access data from modern data stores, organize it into consistent definitions, and deliver it to every application. Like an ORM? Or a middleware? > Cube was designed to work with all…

As part of the Cube team, I have to admit that all descriptions in the sibling comments make a lot of sense. Of course, the "semantic layer" thing is quite known to data engineers/analysts and other data folks in general (they also know things like "metrics store", "headless BI", etc.) but not that well known outside of the data space. Probably, it would be best to describe what are the major use cases Cube is create…

Bunch of links from the website: - https://cube.dev/use-cases/embedded-analytics - https://cube.dev/use-cases/semantic-layer - https://cube.dev/use-cases/real-time-analytics

Re: Cube.js: Headless Semantic Layer

#30

Post by carlineng »

igorlukanin wrote: post #27
rektide wrote: post #2

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. https://carlineng.com/?postid=semantic-layer#blog https://news.ycombinator.com/item?id=35715410 They indeed mentioned Cube.js. Bunch of Malloy links over time: https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDa... And cube.js, https://hn.algolia.com/?dat…

So good you brought Malloy here. I like it quite a bit because the folks really try to innovate (heck, they even have their own data querying syntax to replace SQL). But what I like even more — being part of the Cube team — that the "cons" of existing solutions that Carlin mentions in his blog are actually already solved by Cube. With Cube, Data exploration, ideation on the data model, querying, and bringing the insi…

My understanding of Cube is that iterating on the data model requires the user to (1) write SQL to develop a metric (2) edit YAML or JS config to incorporate the new metric (3) issue API request to Cube server and (4) compare results to raw SQL. Am I mistaken? Does Cube offer a smoother way to do this exploration/iteration?
Post reply on HN