Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's write a treesitter major mode for Emacs

masteringemacs.org

Post reply on HN
Print view View original

31–40 of 86 posts

Re: Let's write a treesitter major mode for Emacs

#31

Post by G3rn0ti »

nanna wrote: post #26
G3rn0ti wrote: post #17

BTW: While Emacs 29.1 comes with "treesitter" built-in, you still need to manually build and install any treesitter language plugin implementing the actual language specific parser. This can be fiddly and frustrating doing it yourself. I had a quick success with using this convenience script: https://github.com/casouri/tree-sitter-module/ . It provides fully-automated builds for the most popular languages (including…

Technically in Emacs 29.1 tree-sitter is still only an optional build option, which a given package maintainer may have 'built in' to your package. It isn't actually a default. If you build it from source you need to pass the --with-tree-sitter flag to ./configure. See: https://www.masteringemacs.org/article/how-to-get-started-tr... What I read from this is that tree-sitter isn't considered quite ready by the Emacs m…

I was lazy. I installed Emacs from Alex Murray's snap:

  sudo snap install emacs
It works w/o problems as it is using snapcraft's "classic" runtime. It comes with both native compilation and tresitter support.

Re: Let's write a treesitter major mode for Emacs

#32

Post by G3rn0ti »

pawelmurias wrote: post #11
craftkiller wrote: post #8

Earlier quoted context omitted.

I'm not the GP, but personally I'd consider syntax highlighting perfectly solved if it was handled in the language server. That way we could have highlighting performed by 100% accurate parsers instead of approximations. It wouldn't be as fast as in-editor simple tree-sitter highlighting, but I don't see any issue with my text appearing uncolored at first and gaining color as my editor polls the language server as I…

You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

> You would only get the top half of your code highlighted because a compiler

Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer) similar to what Treesitter does. It is just not limited to a single file.

That's why I still think Treesitter has the potential to not only improve syntax highlighting but also to simplify language servers or even -- in the long run with some extensions of the the language modes -- replace it altogether.

Re: Let's write a treesitter major mode for Emacs

#33

Post by Difwif »

Is anyone using treesitter with lsp-mode?

I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

Re: Let's write a treesitter major mode for Emacs

#34

Post by PurpleRamen »

raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

There will never be "perfectly" solved problems at that complexity-level. There are always changing requirements and space for improvement. Make it faster, add new features, use new hardware-abilities, follow the flavors of this decade, this is an eternally going game of catching up.

Re: Let's write a treesitter major mode for Emacs

#35

Post by PurpleRamen »

roenxi wrote: post #7
raincole wrote: post #5

Earlier quoted context omitted.

In a perfect world of emacs/tree-sitter, I would imagine tree-sitter to be a single minor mode. You don't need 'java-ts-mode' and 'c-ts-mode'. Just a single 'treesitter-mode' toggle and it will call the treesitter binary to do the hard lifting. I guess the reality just isn't so rosy.

A `treesitter-mode` is a so-so idea, because the way different languages are interacted with is different. The way lisp sexp handle in an IDE is different from Python's syntax. It doesn't make sense to try and have one mode that handles both - it is better to have many modes that interact with the common data structures that treesitter provides and then starts to provide specific convenience functions. Even if all it…

A treesitter-mode would make sense for organizational reasons and user-experience. But under the hood it would be just a proxy-mode which figures out the language and load the appropriate treesitter-sub-modes. And maybe in a more advanced version it could also mix modes for different languages.

Re: Let's write a treesitter major mode for Emacs

#36

Post by intrasight »

jdblair wrote: post #16
wiz21c wrote: post #15

I've been using it a bit but it still not on par with, well, vscode. It tends to be a bit slow on big files (say 10000+ lines) when you open type an fstring in python such as 'print(f"p={' once the open accolade is typed in, it can get noticeably slow. But well, I still love emacs :-)

It's so hard to give up your custom environment and keyboard muscle memory! (25 year emacs user here)

35 years here. 2/3rds of that without any syntax highlighting. My brain is a pretty good parser and I don't like visual distraction. Can you imagine if books had colored syntax highlighting?

I have not embraced some syntax highlighting but with a very subtle color scheme. I tried LSP for a moment but the performance was such that it was a net negative.

I allocate a week over the holiday for "tooling refactors" and I'll probably kick the tires of emacs 29. The big refactor is building a new rig.

Re: Let's write a treesitter major mode for Emacs

#37

Post by olau »

raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

It's a tough problem. Steve Yegge blogged about the complexities involved when he wrote js2-mode:

https://steve-yegge.blogspot.com/2008/03/js2-mode-new-javasc...

I guess comp. sci. people studying languages have been more interested in syntactically valid programs than the opposite.

Re: Let's write a treesitter major mode for Emacs

#38

Post by BaculumMeumEst »

Difwif wrote: post #33

Is anyone using treesitter with lsp-mode? I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

they are mostly used for different things.

lsp (and lsp-mode) are mostly concerned with IDE functionality- go-to definition, show references, displaying project errors in real time without explicitly building, etc.

tree-sitter builds a syntax tree of your source code; its applications are things like syntax highlighting and structural navigation of your code.

there is some overlap in functionality, lsp has somewhat supported mechanisms for syntax highlighting iirc, but they are fairly orthogonal overall

so yes, it makes sense to use them together

Re: Let's write a treesitter major mode for Emacs

#39

Post by Difwif »

BaculumMeumEst wrote: post #38
Difwif wrote: post #33

Is anyone using treesitter with lsp-mode? I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

they are mostly used for different things. lsp (and lsp-mode) are mostly concerned with IDE functionality- go-to definition, show references, displaying project errors in real time without explicitly building, etc. tree-sitter builds a syntax tree of your source code; its applications are things like syntax highlighting and structural navigation of your code. there is some overlap in functionality, lsp has somewhat s…

Interesting, I know LSPs provide me some syntax highlighting and some structural navigation but I haven't compared the two APIs directly. I assumed most LSPs are a superset.

Re: Let's write a treesitter major mode for Emacs

#40

Post by aeonik »

vidarh wrote: post #23
raincole wrote: post #2

I'm not trying to bash Emacs or treesitter or anyone. But I find it mildly amusing that after so many decades, parsing and syntax highlighting aren't a perfectly solved problem, considering programming languages are the most used tools for developers.

Parsing of a correct program is a pretty "solved" problem. But fast enough re-parsing of fragments and recovery from errors is a much more complex problem, that often doesn't have a single correct answer, and it's also a much newer problem in as much as syntax-highlighting is much newer feature, being preceded largely by "offline" pretty-printers with very different constraints. The extent to which modern compilers t…

I wonder why there aren't more tools to allow structural editing like Lisp has. This way the file can never be in an unparsable state.
Post reply on HN