Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's write a treesitter major mode for Emacs

masteringemacs.org

Post reply on HN
Print view View original

51–60 of 86 posts

Re: Let's write a treesitter major mode for Emacs

#51

Post by roenxi »

pulpfictional wrote: post #41
roenxi wrote: post #7

Earlier quoted context omitted.

A `treesitter-mode` is a so-so idea, because the way different languages are interacted with is different. The way lisp sexp handle in an IDE is different from Python's syntax. It doesn't make sense to try and have one mode that handles both - it is better to have many modes that interact with the common data structures that treesitter provides and then starts to provide specific convenience functions. Even if all it…

> GNU Emacs is starting to implement syntax highlighting properly for its system text editor What do you mean?

It is a classic joke. Emacs is often a little behind competing text editors (Vim has better keybindings IMO, things like Visual studio have better IDE support, Atom text editor turns out to have better syntax highlighting, etc, etc).

So, the line of the joke goes, it must not really be a text editor because it is not very good at it and has a wild array of other capabilities. Like "M-x dunnet". Must be an OS.

Of course the reality is that Emacs just copies good ideas from other people. A pretty normal Emacs setup uses Vim keybindings, LSP and now tree-sitter to get a good experience programming.

Re: Let's write a treesitter major mode for Emacs

#52

Post by roenxi »

PurpleRamen wrote: post #35
roenxi wrote: post #7

Earlier quoted context omitted.

A `treesitter-mode` is a so-so idea, because the way different languages are interacted with is different. The way lisp sexp handle in an IDE is different from Python's syntax. It doesn't make sense to try and have one mode that handles both - it is better to have many modes that interact with the common data structures that treesitter provides and then starts to provide specific convenience functions. Even if all it…

A treesitter-mode would make sense for organizational reasons and user-experience. But under the hood it would be just a proxy-mode which figures out the language and load the appropriate treesitter-sub-modes. And maybe in a more advanced version it could also mix modes for different languages.

But you're describing Emacs. Emacs figures out the language and loads the appropriate sub-modes. It even calls them modes. Modes are mixable.

We're in a transition period while everything is rewritten to use tree-sitter. In a few years all the default major modes in Emacs are probably going to be tree-sitter based - unless the maintainers believe them to be better than tree-sitter.

Re: Let's write a treesitter major mode for Emacs

#53

Post by shanusmagnus »

Is there anything that returns a parse tree of an org document? A while ago I wrote some super hacky elisp to navigate around the structure of a giant org mode doc, but it was rickety and terrible and constantly breaking.

Part of this is surely that I don't know wtf I'm doing, but it seemed like there was not an underlying data structure held in memory that you could conveniently query / manipulate, but rather, most of the existing org functionality built some kind of structure each time you did an operation.

Would appreciate any pointers, code examples, tutorials that show how to effectively navigate / manipulate an org structure and have it reflected in the buffer, if there is such a thing.

Re: Let's write a treesitter major mode for Emacs

#54

Post by morelisp »

shanusmagnus wrote: post #53

Is there anything that returns a parse tree of an org document? A while ago I wrote some super hacky elisp to navigate around the structure of a giant org mode doc, but it was rickety and terrible and constantly breaking. Part of this is surely that I don't know wtf I'm doing, but it seemed like there was not an underlying data structure held in memory that you could conveniently query / manipulate, but rather, most…

https://orgmode.org/worg/dev/org-element-api.html

But even with this I found it pretty awful.

Re: Let's write a treesitter major mode for Emacs

#55

Post by rprospero »

aeonik wrote: post #40
vidarh wrote: post #23

Earlier quoted context omitted.

Parsing of a correct program is a pretty "solved" problem. But fast enough re-parsing of fragments and recovery from errors is a much more complex problem, that often doesn't have a single correct answer, and it's also a much newer problem in as much as syntax-highlighting is much newer feature, being preceded largely by "offline" pretty-printers with very different constraints. The extent to which modern compilers t…

I wonder why there aren't more tools to allow structural editing like Lisp has. This way the file can never be in an unparsable state.

I spent a couple of months working with a tool along those lines. Unfortunately, it's one of those issues where it won't work specifically without the galactic emperor mandating these tools under pain of death. A colleague using VS Code would commit code with unbalance parentheses. The tooling was smart enough that it could load the file, even though it wasn't parsable, but there was no structural was to fix the missing closing brace. Adding a closing brace automatically inserted another opening brace, leaving the original mismatch.

Re: Let's write a treesitter major mode for Emacs

#56

Post by nequo »

Difwif wrote: post #33

Is anyone using treesitter with lsp-mode? I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

I use both. In my experience, syntax highlighting with language servers is slower than with tree-sitter.

It stands to reason: a language server often does way more than just incremental parsing of the source code into a concrete syntax tree. By limiting itself to syntax, tree-sitter can be much faster.

Re: Let's write a treesitter major mode for Emacs

#57

Post by morelisp »

G3rn0ti wrote: post #32
pawelmurias wrote: post #11

Earlier quoted context omitted.

You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

> You would only get the top half of your code highlighted because a compiler Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer…

> Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception.

But it is a misconception which framed the discussion, not a misconception of the answerer. The claim was that LSP was

> performed by 100% accurate parsers instead of approximations

Re: Let's write a treesitter major mode for Emacs

#58

Post by Difwif »

nequo wrote: post #56
Difwif wrote: post #33

Is anyone using treesitter with lsp-mode? I see some people say it's possible and use both together but I thought for the most part language servers offer the same set of features, and probably better? My current mental model for how to use them together is that the majority of the languages I quickly read I set up treesitter for speed. For languages I read extensively or write I set up a language server.

I use both. In my experience, syntax highlighting with language servers is slower than with tree-sitter. It stands to reason: a language server often does way more than just incremental parsing of the source code into a concrete syntax tree. By limiting itself to syntax, tree-sitter can be much faster.

Do you keep the lsp semantic tokens capability enabled along with treesitter or only use treesitter for syntax highlights?

Re: Let's write a treesitter major mode for Emacs

#59

Post by imaltont »

shanusmagnus wrote: post #53

Is there anything that returns a parse tree of an org document? A while ago I wrote some super hacky elisp to navigate around the structure of a giant org mode doc, but it was rickety and terrible and constantly breaking. Part of this is surely that I don't know wtf I'm doing, but it seemed like there was not an underlying data structure held in memory that you could conveniently query / manipulate, but rather, most…

While it doesn't properly understand the structure, you can move around pretty well with Imenu or (configured) org-goto. I assume it's also possible to make something for it so that it take nesting into consideration like it does for some programming languages. My org files are only a couple 1000 lines though, so don't know how they perform when it gets larger than that.

Re: Let's write a treesitter major mode for Emacs

#60

Post by imaltont »

G3rn0ti wrote: post #48
yougane wrote: post #47

Earlier quoted context omitted.

Or you can just "M-x treesit-install-language-grammar" then follow the prompts.

I tried that but unsuccessfully. Did not get an error but also no grammar for Typescript or C/C++. Perhaps mileage varies with the language. Edit: I can confirm it works nicely for "javascript". Cool!

I have done typescript successfully. It has another install folder than the default, as the repo has tree-sitter for both tsx and typescript. C/C++ could be a similar situation. The installer should prompt you for it during the setup.
Post reply on HN