Postgres Language Server: Implementing the Parser
1–10 of 47 posts
Re: Postgres Language Server: Implementing the Parser
#2(i'm on the supabase team)
this is an update from the launch here:
https://news.ycombinator.com/item?id=37020610
The focus for the past few months has been getting the Parser right. It has been a lot of work. I'll ping Philipp and get him to join the discussion if there are any questions
We'd also love more contributors, if this sort of thing is up your alley
Re: Postgres Language Server: Implementing the Parser
#3hey, author here. Thanks for posting it!
A bit of background: a few months ago we announced a Postgres language server[0]. A language server adds features like syntax error diagnostic and autocomplete to your editor (vscode, neovim, etc). We have iterated a lot on the parser over the past few months and want to share an update today.
the parser is a core piece of any language server that constructs syntax trees from the raw input string. Usually first an untyped concrete syntax tree (cst) that represents the syntactic structure of the input, and subsequently a typed abstract syntax tree (ast) containing the meaning of the source.
In our implementation, we leverage the actual Postgres parser to-do the heavy lifting. However, the parser is designed to parse executable SQL — not to provide language intelligence. For example, it does not handle incomplete inputs, and outputs just the ast, not the cst. To use it for a language server we had to work around these limitations as good as possible.
While we leverage procedural macros in rust to generate a lot of the repetitive parser code, there remains a portion that requires a bit of manual work. But the groundwork is completed, and we can finally start working on the data model and the actual server next. Our aim is to bring this to a usable state as swiftly as possible.
Huge shout-out to pg_analyze for creating and maintaining libpg_query[1], without which this project would not be possible!
Re: Postgres Language Server: Implementing the Parser
#4I Really like this!
Re: Postgres Language Server: Implementing the Parser
#5I can't wait for this. Just getting out a great AST would be amazing for things like formatters, but I want to build things like a library that knows everywhere you use a table in a particular way or reference a column, or linting that can help you identify problems with the way you do a join or forgot to apply the right filter for multi-tenant queries.
Re: Postgres Language Server: Implementing the Parser
#6When will PG finally start using tasks instead of superheavyweight threads? When will they automatically precompile/hash statements to avoid reparsing? And when materialized views will be automatically updated? Where is an alternative to SQL Server Always On Availability Group released >10 years ago? PostgreSQL is the only database where guys recommend you to configure a separate connection pooler, because their database is unable to handle even mid. size number of connections.
Re: Postgres Language Server: Implementing the Parser
#7When will PG finally start using tasks instead of superheavyweight threads? When will they automatically precompile/hash statements to avoid reparsing? And when materialized views will be automatically updated? Where is an alternative to SQL Server Always On Availability Group released >10 years ago? PostgreSQL is the only database where guys recommend you to configure a separate connection pooler, because their data…
Seems like a rant rather than a comment to the post?
Timescale has continuous aggregates that might be interesting, writing a table built by a trigger isn't hard, and some might say having a separate connection pooler is a feature rather than a bug... Idk enough about SQL server for the group part.
Re: Postgres Language Server: Implementing the Parser
#8When will PG finally start using tasks instead of superheavyweight threads? When will they automatically precompile/hash statements to avoid reparsing? And when materialized views will be automatically updated? Where is an alternative to SQL Server Always On Availability Group released >10 years ago? PostgreSQL is the only database where guys recommend you to configure a separate connection pooler, because their data…
it's a great list. No database is perfect, and this is a good set of features that are missing from Postgres
that said, the flexibility/extensibility of Postgres still makes it a great choice. You could easily point at other databases and produce a list similar (or longer) than this.
Re: Postgres Language Server: Implementing the Parser
#9When will PG finally start using tasks instead of superheavyweight threads? When will they automatically precompile/hash statements to avoid reparsing? And when materialized views will be automatically updated? Where is an alternative to SQL Server Always On Availability Group released >10 years ago? PostgreSQL is the only database where guys recommend you to configure a separate connection pooler, because their data…
I'll take vertical sharding until no longer possible and then dropping all further requests over touching Always On Availability Groups ever again
Re: Postgres Language Server: Implementing the Parser
#10I can't wait for this. Just getting out a great AST would be amazing for things like formatters, but I want to build things like a library that knows everywhere you use a table in a particular way or reference a column, or linting that can help you identify problems with the way you do a join or forgot to apply the right filter for multi-tenant queries.
that is definitely the goal, both a formatter and a linter. we want to add something like squawk and plpgsql_check directly to the language server, so you get eslint-like dx. with both the ast and the database schema at hand, you can basically add any rule you like.
and this is far out, but eventually we are maybe even able to combine the language server with declarative schema management and have go-to-definition etc working.
[0] https://github.com/sbdchd/squawk/tree/master [1] https://github.com/okbob/plpgsql_check