Crux SQL
juxt.pro
Crux SQL
1–10 of 36 posts
Re: Crux SQL
#2Does this mean I can use tools like https://github.com/metabase/metabase with crux? Awesome if so!
Re: Crux SQL
#3I learned datalog through Crux and I really enjoyed it. After not very long, it was more intuitive and easier to reason about that similar sql.
Re: Crux SQL
#4Does this mean I can use tools like https://github.com/metabase/metabase with crux? Awesome if so!
I don't believe Metabase supports the Calcite SQL dialect (https://github.com/metabase/metabase/issues/6230), which is what Crux is using for the SQL layer. So I believe the answer is no - but I'm not an expert here, so don't take this answer as definitive.
Re: Crux SQL
#5Does this mean I can use tools like https://github.com/metabase/metabase with crux? Awesome if so!
I don't believe Metabase supports the Calcite SQL dialect ( https://github.com/metabase/metabase/issues/6230 ), which is what Crux is using for the SQL layer. So I believe the answer is no - but I'm not an expert here, so don't take this answer as definitive.
As it happens there was a comment on a related issue about Dremio support [0] (which also uses Calcite) where someone shared that they got a Dremio driver working. I was able to fork their driver and get Metabase working against Crux in time for a live demo of the crux-sql module back in May [1].
I just switched the forked driver repo to public if anyone wants to test it out [2]. The Metabase driver docs are pretty straightforward to get things running. There's definitely work to be done though and I didn't get very deep into it really - but I hope to pick it up again soon!
[0] https://github.com/metabase/metabase/issues/5562
[1] https://youtu.be/StXLmWvb5Xs?t=996
[2] https://github.com/crux-labs/crux-metabase-driver
(I work on Crux :)
EDIT: it's also worth a mention that the linked issue #6230 discusses a lot of problems deriving from Druid's lack of support for prepared statements, but Crux and Dremio don't have that limitation. Although reading the most recent comment it looks like Druid may have overcome that hurdle now, so with a bit of luck there might now be more traction to get mainline Metabase support for a generic Calcite driver!
Re: Crux SQL
#6Any relation to the CRUX Linux distro? Or is this just an unfortunate overlap of names?
Re: Crux SQL
#7I learned datalog through Crux and I really enjoyed it. After not very long, it was more intuitive and easier to reason about that similar sql.
I'm trying to learn datalog casually, there's lots of queries that I still don't know how they translate
I.e. group by, window aggregates, order by, limit
I'd be interested in seeing the equivalent datalog that Crux makes given some of those statements for learning what efficient datalog looks like for some of those problems
Re: Crux SQL
#8Anyone with experience has some detailed info about the limitation, trade-offs and problems of using Datalog instead of SQL?
Datalog seems like a natural choice as declarative query language but has not yet become mainstream. I wonder why?
Re: Crux SQL
#9Anyone with experience has some detailed info about the limitation, trade-offs and problems of using Datalog instead of SQL? Datalog seems like a natural choice as declarative query language but has not yet become mainstream. I wonder why?
Using Crux's Datalog query language from Clojure is great. Normally if your query building library is giving you a data-like interface, then you're relying on some abstraction that will inevitably leak in places (and let's not talk about ORMs), but since Crux really does natively use EDN (Clojure's data syntax, roughly analogous to JSON) for its query language, you've eliminated an entire layer of complexity in your application, and you also have the entire programming language at your disposal for compositionally building queries.
That's not to say that using the SQL API is a bad idea, but if you're using Crux from Clojure, you'd be missing out on some stuff.
Re: Crux SQL
#10I learned datalog through Crux and I really enjoyed it. After not very long, it was more intuitive and easier to reason about that similar sql.
I'm trying to learn datalog casually, there's lots of queries that I still don't know how they translate I.e. group by, window aggregates, order by, limit I'd be interested in seeing the equivalent datalog that Crux makes given some of those statements for learning what efficient datalog looks like for some of those problems
You always have the option to wire together multiple Datalog queries using Clojure/JVM-lang-of-choice (the N+1 query problem doesn't affect Crux when embedded), but we are currently analysing the possibilities for a higher-level query API, potentially in a similar vein to Mongo's pipelines [0], where a sequence of "stages" is defined. In the ideal case there would be _closure_ in the intermediate steps of a pipeline such that the output to any given stage (including Datalog queries) is a fully-fledged graph in its own right.
As for your specific examples, Crux already implements an order-by+limit+offset combination that automatically spills to disk if needed, but for efficient pagination you would probably want to maintain additional pre-sorted value ranges. For basic aggregation we have an alpha API decorator that composes Clojure transducers to great effect [1].
[0] https://docs.mongodb.com/manual/reference/operator/aggregati...
[1] https://github.com/crux-labs/crux-decorators/blob/master/tes...