Live data from Hacker News

Sqlfluff the SQL Linter for Humans

sqlfluff.com

31–40 of 95 posts

Re: Sqlfluff the SQL Linter for Humans

#31

Earlier quoted context omitted.

Or the ultimate pedant (pedultimate?) order which matches the SQL order of operations (which for some queries can be illuminating): from ... where ... group ... having ... select ... order by ... limit ...

Standard SQL and most dialects enforce that in the syntax anyway no? Edit: misread and select was in the middle in that comment

No, all databases support the common order of SELECT ... FROM ... WHERE ....

Re: Sqlfluff the SQL Linter for Humans

#32

I wanted to use this linter, but they don't support many things that are used in real production code. By example, a simple postgres function like this one can't be parsed: CREATE OR REPLACE FUNCTION public.setof_test() RETURNS SETOF text LANGUAGE sql STABLE STRICT AS $function$ select unnest(array['hi', 'test']) $function$ ;

The code looks really great [1], very declarative, looks like it should be pretty easy to add better PostgreSQL support. I also wonder if how easy it would be to plug it into VS Code, and build in an auto formatter (♥ω♥ ). At a glance it looks like it's parsing AST, which is really hard to find for SQL, so auto formatter could be possible to build from this. I notice SQLite could use some improving too, seems like a great project to coalesce around.

[1] https://github.com/sqlfluff/sqlfluff/blob/main/src/sqlfluff/...

Re: Sqlfluff the SQL Linter for Humans

#33
post #11

There surely is a need for a sql linter, but if it wants to be strict about syntax style, it should better familiarize itself with not-noob styles first. Eg. some people like to align the clauses vertically and use a leading, not trailing, comma. select a , b , count(*) as count from table where cond1 and cond2 group by a, b order by count desc

If SQL would just allow a trailing comma on the last column, none of this leading comma shit would be necessary. I still use trailing commas, because code readability is more important that writeability IMO, but the fact that I'm forced to sympathize with such an ugly shit sandwich is really frustrating.

Can't sympathize with the clause alignment though. And I can't figure out why someone who does leading commas for editability would chose a style that forces you to use different indentation levels for different lines. Oh wait, you wanna make that join a left join? Time to go re-indenting my whole damn query...

It's not even an improvement in readability...who actually reads text like it's a column format? It's not a spreadsheet.

Re: Sqlfluff the SQL Linter for Humans

#34

Earlier quoted context omitted.

Standard SQL and most dialects enforce that in the syntax anyway no? Edit: misread and select was in the middle in that comment

No, all databases support the common order of SELECT ... FROM ... WHERE ....

You're right, I somehow missed that select was in a different order.

Re: Sqlfluff the SQL Linter for Humans

#35
post #32

I wanted to use this linter, but they don't support many things that are used in real production code. By example, a simple postgres function like this one can't be parsed: CREATE OR REPLACE FUNCTION public.setof_test() RETURNS SETOF text LANGUAGE sql STABLE STRICT AS $function$ select unnest(array['hi', 'test']) $function$ ;

The code looks really great [1], very declarative, looks like it should be pretty easy to add better PostgreSQL support. I also wonder if how easy it would be to plug it into VS Code, and build in an auto formatter (♥ω♥ ). At a glance it looks like it's parsing AST, which is really hard to find for SQL, so auto formatter could be possible to build from this. I notice SQLite could use some improving too, seems like a…

Auto-formatter is already built in for some simple violations: https://docs.sqlfluff.com/en/stable/cli.html#sqlfluff-fix

Re: Sqlfluff the SQL Linter for Humans

#36
post #5

This is a nice idea, but my problem with most of these tools is that they work with pure SQL/JS/whatever files, which isn't always the case. For example, if you use something like myBatis, you'll probably build the SQL dynamically, which could be stored in a mapper XML file: https://mybatis.org/mybatis-3/dynamic-sql.html Not only that, but certain frameworks have you embedding SQL inside of the main application code,…

You're totally right that there isn't any consistency in this space at the moment. It's part of the justification for tools like SQLFluff starting to align people on a common way of writing SQL. None of the incumbent players have an incentive to do that.

Re: Sqlfluff the SQL Linter for Humans

#37
post #18

Earlier quoted context omitted.

Makes you wonder if this tool would be better a code formatter rather than linter?

It seems to have a formatter option in the `fix` argument you can give it. The authors should probably be selling it on that point since it seems to be its strength, it doesn't have many rules along the lines of the static analysis that people look for in linters. Some people (tech leads, CTOs) care a lot about formatting but I wouldn't make teammates go through a linter for that, it's a rather surface concern.

The reason that those people care about formatting is that it matters when the code is used to communicate with other people, not just with machines. Having a consistent style within the team, means the team can communicate more easily - even if the net effect for an individual working on a codebase alone is minimal.

Re: Sqlfluff the SQL Linter for Humans

#38
post #17

Sorry - didn't get a chance to go through the documentation much yet... A problem we are looking to catch early in the development process is for things we don't want folks to do, even if they are technically correct. For example, dropping a column or altering the datatype/nullability in such a way that the table goes into reorg pending. I know we can write our own regex to look for that specific syntax, but I've not…

It's pretty new functionality in sqlfluff, but it now supports user defined plugins for org-specific rules if you want to forbid something more obscure. Documentation is sketchy, but you can see the proof of concept here: https://github.com/sqlfluff/sqlfluff/tree/main/plugins/sqlfl...

Re: Sqlfluff the SQL Linter for Humans

#39
post #11

There surely is a need for a sql linter, but if it wants to be strict about syntax style, it should better familiarize itself with not-noob styles first. Eg. some people like to align the clauses vertically and use a leading, not trailing, comma. select a , b , count(*) as count from table where cond1 and cond2 group by a, b order by count desc

Most of the SQLFluff style was based around the Mozilla SQL style guide and the dbt labs style guide: https://docs.telemetry.mozilla.org/concepts/sql_style.html https://github.com/dbt-labs/corp/blob/master/dbt_style_guide...

The GitLab SQL style guide isn't far off either: https://about.gitlab.com/handbook/business-technology/data-t...

Rather than supporting _all_ ways of formatting SQL, the aim of the tool was to fairly flexibly support the most mainstream ones with a view to becoming more opinionated in future to converge on a more unified style in future. Similar to what black has done with python.

Re: Sqlfluff the SQL Linter for Humans

#40
I think this is primarily aimed at analytics and data modelling SQL, the type of SQL in dbt [], though I'd be happily corrected by the team.

As far as I understand, this has been quite a long time coming, and is far from trivial due to the diversity of SQL dialect.

My use case would be to have this attached to a dbt project that ensures the dbt SQL files all confirm to a passable degree of consistency when working across a team. Especially as a CICD automated bot action thing on github.

[] https://www.getdbt.com/

Post reply on HN