Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's write a treesitter major mode for Emacs

masteringemacs.org

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Let's write a treesitter major mode for Emacs

#41

Post by pulpfictional »

roenxi wrote: post #7
raincole wrote: post #5

Earlier quoted context omitted.

In a perfect world of emacs/tree-sitter, I would imagine tree-sitter to be a single minor mode. You don't need 'java-ts-mode' and 'c-ts-mode'. Just a single 'treesitter-mode' toggle and it will call the treesitter binary to do the hard lifting. I guess the reality just isn't so rosy.

A `treesitter-mode` is a so-so idea, because the way different languages are interacted with is different. The way lisp sexp handle in an IDE is different from Python's syntax. It doesn't make sense to try and have one mode that handles both - it is better to have many modes that interact with the common data structures that treesitter provides and then starts to provide specific convenience functions. Even if all it…

> GNU Emacs is starting to implement syntax highlighting properly for its system text editor What do you mean?

Re: Let's write a treesitter major mode for Emacs

#42

Post by ghosty141 »

aeonik wrote: post #40
vidarh wrote: post #23

Earlier quoted context omitted.

Parsing of a correct program is a pretty "solved" problem. But fast enough re-parsing of fragments and recovery from errors is a much more complex problem, that often doesn't have a single correct answer, and it's also a much newer problem in as much as syntax-highlighting is much newer feature, being preceded largely by "offline" pretty-printers with very different constraints. The extent to which modern compilers t…

I wonder why there aren't more tools to allow structural editing like Lisp has. This way the file can never be in an unparsable state.

This is what treesitter enables!

Re: Let's write a treesitter major mode for Emacs

#43

Post by mbork_pl »

olau wrote: post #37
raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

It's a tough problem. Steve Yegge blogged about the complexities involved when he wrote js2-mode: https://steve-yegge.blogspot.com/2008/03/js2-mode-new-javasc... I guess comp. sci. people studying languages have been more interested in syntactically valid programs than the opposite.

Well, I bet most syntactically valid programs are not interested in comp-sci people!

Re: Let's write a treesitter major mode for Emacs

#44

Post by Difwif »

BaculumMeumEst wrote: post #38
Difwif wrote: post #33

Is anyone using treesitter with lsp-mode? I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

they are mostly used for different things. lsp (and lsp-mode) are mostly concerned with IDE functionality- go-to definition, show references, displaying project errors in real time without explicitly building, etc. tree-sitter builds a syntax tree of your source code; its applications are things like syntax highlighting and structural navigation of your code. there is some overlap in functionality, lsp has somewhat s…

Quick follow up, looked into the LSP spec and it does define "SemanticTokens" since 3.16

https://microsoft.github.io/language-server-protocol/specifi...

Obviously it depends on the LSP implementation but it looks expressive enough to replace treesitter?

Re: Let's write a treesitter major mode for Emacs

#45

Post by mbork_pl »

raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

Well, there are more problems like that. You'd think diffing is a solved problem, and yet we still struggle with syntax-aware diffs (I use difftastic, which is great, but doesn't always work well, and is under constant development).

Re: Let's write a treesitter major mode for Emacs

#46

Post by nequo »

aeonik wrote: post #40
vidarh wrote: post #23

Earlier quoted context omitted.

Parsing of a correct program is a pretty "solved" problem. But fast enough re-parsing of fragments and recovery from errors is a much more complex problem, that often doesn't have a single correct answer, and it's also a much newer problem in as much as syntax-highlighting is much newer feature, being preceded largely by "offline" pretty-printers with very different constraints. The extent to which modern compilers t…

I wonder why there aren't more tools to allow structural editing like Lisp has. This way the file can never be in an unparsable state.

You may be aware but the author of TFA also has a tree-sitter based minor mode called Combobulate for exactly that:

https://www.masteringemacs.org/article/combobulate-structure...

There is also evil-textobj-tree-sitter for tree-sitter based text objects for Evil mode:

https://github.com/meain/evil-textobj-tree-sitter

Re: Let's write a treesitter major mode for Emacs

#47

Post by yougane »

G3rn0ti wrote: post #17

BTW: While Emacs 29.1 comes with "treesitter" built-in, you still need to manually build and install any treesitter language plugin implementing the actual language specific parser. This can be fiddly and frustrating doing it yourself. I had a quick success with using this convenience script: https://github.com/casouri/tree-sitter-module/ . It provides fully-automated builds for the most popular languages (including…

Or you can just "M-x treesit-install-language-grammar" then follow the prompts.

Re: Let's write a treesitter major mode for Emacs

#48

Post by G3rn0ti »

yougane wrote: post #47
G3rn0ti wrote: post #17

BTW: While Emacs 29.1 comes with "treesitter" built-in, you still need to manually build and install any treesitter language plugin implementing the actual language specific parser. This can be fiddly and frustrating doing it yourself. I had a quick success with using this convenience script: https://github.com/casouri/tree-sitter-module/ . It provides fully-automated builds for the most popular languages (including…

Or you can just "M-x treesit-install-language-grammar" then follow the prompts.

I tried that but unsuccessfully. Did not get an error but also no grammar for Typescript or C/C++. Perhaps mileage varies with the language.

Edit: I can confirm it works nicely for "javascript". Cool!

Re: Let's write a treesitter major mode for Emacs

#49

Post by [deleted] »

mickeyp wrote: post #13
raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

I clarified exactly why it is the way it is here: https://www.masteringemacs.org/article/tree-sitter-complicat... And also why using LSP to furnish your editor with highlight markers is an inelegant solution for many languages.

[deleted]

Re: Let's write a treesitter major mode for Emacs

#50

Post by gpderetta »

G3rn0ti wrote: post #32
pawelmurias wrote: post #11

Earlier quoted context omitted.

You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

> You would only get the top half of your code highlighted because a compiler Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer…

But some do. Most notably clangd. Still clangd tries parse broken code using recovery heuristics (which can still be useful in a compiler to produce decent error messages).
Post reply on HN