I've been very impressed with sqlglot, and am looking forward to trying this feature. The only issue I've had with sqlglot is transpiling for use with a specific spark version: in my experience Spark is not great about surfacing obvious 'not registered' errors when a function isn't supported (especially in >=2.4). I ran into this with width_bucket, which is only in the most recent release. I am curious whether there'…
Semantic Diff for SQL
11–20 of 34 posts
Re: Semantic Diff for SQL
#12Very nice/interesting.
Somewhat related question and apologies if this is already stated in the documentation (it's rather dense and I haven't had the time to read through it completely)...
Can you use sqlglot to create custom DDL dialects that have custom first class objects? For instance, if I want to build a custom SQL/DDL/DML dialect that had a new kind of object such as a "pipe", "kitchensink", etc, would sqlglot be a good tool to use?
I've tried playing around with Apache Calcite, but it lost me pretty quickly since the examples to customize/extend DDLs were quite lacking in my opinion.
Re: Semantic Diff for SQL
#13Very nice/interesting. Somewhat related question and apologies if this is already stated in the documentation (it's rather dense and I haven't had the time to read through it completely)... Can you use sqlglot to create custom DDL dialects that have custom first class objects? For instance, if I want to build a custom SQL/DDL/DML dialect that had a new kind of object such as a "pipe", "kitchensink", etc, would sqlglo…
Yes. SQLGlot is very customizable and you can pretty much override everything. It kind of needs to be flexible because even common sql dialects vary greatly.
Here's an example of how we use SQLGlot to output raw Python code directly from SQL.
https://github.com/tobymao/sqlglot/blob/main/sqlglot/executo...
Re: Semantic Diff for SQL
#14Interesting, will give sqlglot a look when we get to adding SQL support in DiffLens [https://github.com/marketplace/difflens]. Or perhaps DiffLens can just use sqlglot :) Either way we're very happy to see another semantic diff tool.
P.S: We work on DiffLens. It currently supports TS, JS, CSS and text diffs. We're working on making a VS Code extension currently
Re: Semantic Diff for SQL
#15Very nice/interesting. Somewhat related question and apologies if this is already stated in the documentation (it's rather dense and I haven't had the time to read through it completely)... Can you use sqlglot to create custom DDL dialects that have custom first class objects? For instance, if I want to build a custom SQL/DDL/DML dialect that had a new kind of object such as a "pipe", "kitchensink", etc, would sqlglo…
Yes. SQLGlot is very customizable and you can pretty much override everything. It kind of needs to be flexible because even common sql dialects vary greatly. Here's an example of how we use SQLGlot to output raw Python code directly from SQL. https://github.com/tobymao/sqlglot/blob/main/sqlglot/executo...
Very nice, and exactly along the lines of what I was thinking... I want to be able to create a custom SQL dialect that can output some code.
Thanks very much, looking forward to spending some time reading through all of this!
Re: Semantic Diff for SQL
#16this is very cool, but I believe this bit of the README is incorrect:
Text-based diff tools such as git diff, when applied to a code base, have certain limitations. First, they can only detect insertions and deletions, not movements or updates of individual pieces of code.
git diff can detect movements. looking at my .gitconfig, I think it's the "frag = magenta" line.
Re: Semantic Diff for SQL
#17What about difftastic?
Difftastic seems like a really cool tool. There are a few reasons, however, why it doesn't apply well to use cases I had in mind: 1. It's in JS and not Python. These days a common data (including data tooling) stack revolves around Python and fitting JS into this ecosystem is not straightforwad. 2. Limited dialect support. As far as I can see it only supports "PostgreSQL flavor" (not sure what exactly is meant by "fl…
Re: Semantic Diff for SQL
#18this is very cool, but I believe this bit of the README is incorrect: Text-based diff tools such as git diff, when applied to a code base, have certain limitations. First, they can only detect insertions and deletions, not movements or updates of individual pieces of code. git diff can detect movements. looking at my .gitconfig, I think it's the "frag = magenta" line.
Post it or link it so we can better understand!
Re: Semantic Diff for SQL
#19I thought this was going to be something else like being able to tell that a rewritten query returns the same set of rows, but with potentially a very different query plan. E.g. dependent EXISTS subquery vs IN subquery.
This exists for Postgres, let me try to find the name of the tool EDIT: I can't find it, I searched for 30 mins, I promise this exists though. If anyone else can remember the name of it, please post.
[deleted]
Re: Semantic Diff for SQL
#20I thought this was going to be something else like being able to tell that a rewritten query returns the same set of rows, but with potentially a very different query plan. E.g. dependent EXISTS subquery vs IN subquery.
This exists for Postgres, let me try to find the name of the tool EDIT: I can't find it, I searched for 30 mins, I promise this exists though. If anyone else can remember the name of it, please post.
Pg_query maybe?
https://pganalyze.com/blog/pg-query-2-0-postgres-query-parse...
Scroll down to the section marked "Fingerprints in...".