41–47 of 47 posts
I like the idea of a semantic Layer but don't think defining it in yaml is the right way to go about it. Semantic Layer needs proper language and tooling support which Malloy provides.
Yeah, we still have them, MDX before and now DAX :) I curate some more on here in case of interest: https://www.ssp.sh/brain/data-modeling-languages .
Earlier quoted context omitted.
From the article: > There's a lot of information out there, including from myself about the history and rise [2022], comparing it to an MVC-like approach, or explaining its capabilities. That's why in this article I focus on the why and showcase how to use it in a practical example in the next chapter. [1] https://www.ssp.sh/blog/rise-of-semantic-layer-metrics/ [2] https://cube.dev/blog/exploring-the-semantic-layer-t…
I started writing another comment and after looking at your links and was about to quote the definition at the top of "The Rise of the Semantic Layer" as a suggestion, but I realized that it actually isn't that far off, information-wise from the definition youve provided here. So I took a step back and tried to think about why one "feels" to a reader more like a definition than the other. I think it comes down to phr…
> A semantic layer is an interface to data stores that is designed to be queryable in terms relevant and familiar to those with knowledge of the business domain.
Sounds good to me, but I think it's too simplified. A semantic layer, IMO, does more. See Julian Hyde's definition, which is also similar to mine, and more involved as well:
> A semantic layer, also known as a metrics layer, lies between business users and the database, and lets those users compose queries in the concepts that they understand. It also governs access to the data, manages data transformations, and can tune the database by defining materializations.
> Like many new ideas, the semantic layer is a distillation and evolution of many old ideas, such as query languages, multidimensional OLAP, and query federation.
I appreciated your feedback. Will think a little more about it.
Earlier quoted context omitted.
It's a lot more. A SQL VIEW is just a saved query, where a semantic layer defines the shared meaning of the data, and helps enforce consistent metrics, joins, and logic across tools. You'd be surprised at how many ways "active customer" can be represented as SQL.
Doesn't a view do that? create view active_cx as select * from customer join audit_events using(...) join ... where -- active condition -- use active_cx wherever select ... from orders join active_cx using(...) where ts > start_of_month() group by active_cx.id
As one of the consumers of a "semantic layer" for many years now, I am firmly convinced that a "single source of truth" must either be useless or a lie.
Ok, the DBA has produced some joins that I can count up to decide how many "customers" we have. We immediately have the issue that a "customer count" from the semantic layer cannot always be the meaningful or relevant figure. In my experience, outside of the exllicit context it was written it, it cannot be the correct figure. So, I have my single source of truth customer count, but my revenue per customer needs to to use a different count that's slightly off. Another analyst needs to produce customer calls to our call center and that uses a slightly different definition. And so on, until the semantic layer is just a special database for pre-defined executive KPI dashboards and no more.
I like the idea of a semantic Layer but don't think defining it in yaml is the right way to go about it. Semantic Layer needs proper language and tooling support which Malloy provides.
are you using Malloy in prod? Do you know of anyone? I remember the Malloy hype during Data Council in 2023. Haven't heard about it since.
Impressive! An entire article about semantic layers, artfully avoids ever defining what a semantic layer is. Let me take a swipe at it: a semantic layer helps express queries and their results in terms the end-consumers will care about / prefer to reason in, instead of whatever extremely correct and efficient atrocities the database nerds came up with. Did I get that right?
Julian Hyde (Apache Calcite, Google) gave a crisp presentation on this and how SQL could express 'measures' to bridge the gap: https://communityovercode.org/wp-content/uploads/2023/10/mon... > A semantic layer, also known as a metrics layer, lies between business users and the database, and lets those users compose queries in the concepts that they understand. It also governs access to the data, manages data transfor…
Impressive! An entire article about semantic layers, artfully avoids ever defining what a semantic layer is. Let me take a swipe at it: a semantic layer helps express queries and their results in terms the end-consumers will care about / prefer to reason in, instead of whatever extremely correct and efficient atrocities the database nerds came up with. Did I get that right?
From the article: > There's a lot of information out there, including from myself about the history and rise [2022], comparing it to an MVC-like approach, or explaining its capabilities. That's why in this article I focus on the why and showcase how to use it in a practical example in the next chapter. [1] https://www.ssp.sh/blog/rise-of-semantic-layer-metrics/ [2] https://cube.dev/blog/exploring-the-semantic-layer-t…