Does GitHub currently use tree-sitter for syntax highlighting? If yes, are the libraries open-source? Thanks :)
Tree-sitter: an incremental parsing system for programming tools
121–130 of 138 posts
Re: Tree-sitter: an incremental parsing system for programming tools
#122Where is the SQL parser? Any specific reason why is it missing (not even started)?
My team is only writing tree-sitter parsers as part of working on GitHub developer productivity features like Code Navigation. So the short version is that we (i.e., my team at GitHub) haven't written a tree-sitter parser for SQL because we haven't targeted SQL for Code Nav support yet.
That said, this is exactly why we've released tree-sitter as an open-source project. That way there's no need for anyone to be blocked on my team finding the time to work on an SQL parser. Most extant tree-sitter parsers [1] have been developed by external language communities, and not by the core tree-sitter maintainers.
(Also note that SQL is a particularly wrinkly language, since there are so many different dialects. Are you looking for an ANSI SQL parser? A MySQL SQL parser? One that covers all of them to some degree?)
[1] https://tree-sitter.github.io/tree-sitter/#available-parsers
Re: Tree-sitter: an incremental parsing system for programming tools
#123This is really cool, I 100% agree that as programmers we’re editing and thinking in terms of ASTs. It just happens that text is a high density way to represent those ASTs.
I’m going to play with this and see if I can make a generic language server for vscode that works across languages. Unless someone has already done that.
What would be really cool is that tree-sitter (or a sister package) that provides incremental formatting primitives across languages.
The closest language agnostic formatter that comes to mind is prettier.js with its extensions.
incremental parser —> language server -> formatter across languages would be super rad.
Re: Tree-sitter: an incremental parsing system for programming tools
#124Earlier quoted context omitted.
Tooting my own horn, Emacs’ csharp-mode[1] is undergoing a rewrite to be 100% based on tree-sitter rather than regexps. The new code runs way faster and is so much nicer to work with. Once all the kinks are gone, I can’t imagine going back. [1] https://github.com/emacs-csharp/csharp-mode/blob/master/csha...
This looks awesome! Do you mind elaborating on any kinks which you found?
A few:
- indentation may be fine for a final doc, but not always while editing. Especially for new lines starting new code-blocks.
- adding new syntax not already known by tree-sitter requires up streaming to at least 2 repos before we can use it in a released version of our package. This can feel less hands on and slow than working in a single repo where you have full control.
No super-biggies yet though.
Re: Tree-sitter: an incremental parsing system for programming tools
#125Does GitHub currently use tree-sitter for syntax highlighting? If yes, are the libraries open-source? Thanks :)
> Does GitHub currently use tree-sitter for syntax highlighting?
For some languages, yes. https://news.ycombinator.com/item?id=26227214
> If yes, are the libraries open-source?
They are! tree-sitter itself is open-source [1], as are all of the language parsers we've listed on the homepage [2]. The syntax highlighting support is documented here [3].
[1] https://github.com/tree-sitter/tree-sitter
[2] https://tree-sitter.github.io/tree-sitter/#available-parsers
[3] https://tree-sitter.github.io/tree-sitter/syntax-highlightin...
Re: Tree-sitter: an incremental parsing system for programming tools
#126I tried to use this to ease the front end work load of students in a compiler project (building a C compiler) for a University course, so that the project could be focused on the more interesting middle and back end parts of the compiler. However, reported bugs in the C grammar that saw no activity at all [1] made this impossible. From this small sample of experiences, I was left with the impression that Tree Sitter…
Hi there! You're right that the C grammar in particular is one that could use some love. C is not one of the languages that we're syntax highlighting with tree-sitter yet, nor is it one of the languages that we support Code Navigation for. That means that my team has had to prioritize their work in other places, and no community members have stepped up to take over or help out with maintenance of the C grammar. Not a satisfying answer, I realize, but an honest one.
Re: Tree-sitter: an incremental parsing system for programming tools
#127While we're in this discussion: Say I want to implement "SQL" for my app (if you've used Jira, I want to make my own JQL). Is this the tool for that? I'm looking for something much simpler than ANTLR.
Should be able to create your own grammar.js, it’s actually much simpler than ANTLR.
Re: Tree-sitter: an incremental parsing system for programming tools
#128Wrote tree-sitter-svelte. Was a good experience. I am also writing a programming language of my own similar to TypeScript and I am using tree-sitter for the same. Its a delight to work with it. Removes a lot of the worries.
You have your code/demo on GitHub, would love to play with it.
Re: Tree-sitter: an incremental parsing system for programming tools
#129Earlier quoted context omitted.
The idea is pretty awesome, but my eyes nearly rolled out of my head from the needless condescension at the beginning.
it's just how old people talk. Rob Pike speaks of syntax colouring in the same way - he quotes the Bible. I don't see it as condescension though. I think it's just a way of speaking.
[deleted]
Re: Tree-sitter: an incremental parsing system for programming tools
#130Earlier quoted context omitted.
This looks awesome! Do you mind elaborating on any kinks which you found?
A few: - indentation may be fine for a final doc, but not always while editing. Especially for new lines starting new code-blocks. - adding new syntax not already known by tree-sitter requires up streaming to at least 2 repos before we can use it in a released version of our package. This can feel less hands on and slow than working in a single repo where you have full control. No super-biggies yet though.
When tree-sitter reaches 1.0 [0], it may be possible to eliminate the tree-sitter-langs upstream, or both.