Speeding up VSCode extensions in 2022 • Hacker News Board

Re: Speeding up VSCode extensions in 2022

#21

Post by eyelidlessness » Fri, Jan 28, 2022, 6:38 PM UTC

tsar9x wrote: post #5

duped wrote: post #2

The tokenization speed issue is already addressed by the language server protocol, which delegates syntax highlighting to language servers via the "semantic tokens" API. Language servers can choose to implement this however they want, and the API allows for incremental additions. I think this is a much better approach that baking tree sitter into VS Code and continuing to use TextMate grammars (or tree sitter specifi…

> I think this is a much better approach that baking tree sitter into VS Code they're implementing both, with tree sitter being 'dumb' version of LSP syntax highlighting: https://github.com/microsoft/vscode-anycode

I was curious about the implementation, and was surprised to find:

- they’re importing query files, with a .scm extension, in TypeScript

- they’re using esbuild to handle those imports

It’s surprising to me that the Microsoft, who created and maintains TypeScript, is using an alternative TypeScript compiler… in part to query semantic information from TypeScript.

Re: Speeding up VSCode extensions in 2022

#22

Post by eyelidlessness » Fri, Jan 28, 2022, 6:41 PM UTC

egg1 wrote: post #12

The more I read about VSCode and Electron apps in general, the more I'm convinced that all this effort is akin to putting lipstick on a pig. Modern CPU designs have all converged on multi-core as the solution to increasing performance, and yet on the software side of things... we've turned all of our desktop applications into Chrome instances running single-threaded event loops. That these extensions are even more po…

It would be utterly shocking to me if VSCode isn’t using several worker threads for LSP, extensions, etc.

Re: Speeding up VSCode extensions in 2022

#23

Post by speedgoose » Fri, Jan 28, 2022, 6:44 PM UTC

Tree-sitter is interesting. I see that it is used by the Github Copilot extension for VS-Code, which is the slowest on my machine because it somehow contains megabytes of minified javascript and webassembly to call a remote API.

Re: Speeding up VSCode extensions in 2022

#24

Post by fcurts » Fri, Jan 28, 2022, 6:51 PM UTC

da39a3ee wrote: post #17

fcurts wrote: post #11

Earlier quoted context omitted.

It uses heuristics to derive semantic information from the parse tree without doing a full semantic analysis.

Is "semantic analysis" in the sense you used it similar to "type checking"?

Semantic analysis is a broader term that (for languages with a type checker) includes type checking. https://cs.lmu.edu/~ray/notes/semanticanalysis/

Re: Speeding up VSCode extensions in 2022

#25

Post by Tyriar » Fri, Jan 28, 2022, 6:53 PM UTC

eyelidlessness wrote: post #22

egg1 wrote: post #12

The more I read about VSCode and Electron apps in general, the more I'm convinced that all this effort is akin to putting lipstick on a pig. Modern CPU designs have all converged on multi-core as the solution to increasing performance, and yet on the software side of things... we've turned all of our desktop applications into Chrome instances running single-threaded event loops. That these extensions are even more po…

It would be utterly shocking to me if VSCode isn’t using several worker threads for LSP, extensions, etc.

VS Code team member here. The diagram in the article is a little wrong, but the basics of it are:

- The "main process" which manages the windows (renderer processes)

- The renderer process" contains the UI thread for each window, the renderer process can have its own worker threads

- The extension host loads extensions in proc, extensions are free to create their own threads/processes. The separate process for extensions protects extensions from freezing the renderer

- Various other processes that live off either the main process or the "shared process", such as the pty host for terminals which enables the terminal reconnection feature when reloading a window (also file watcher, search process)

We've been shuffling where processes are launched from recently but the actual processes and their purpose probably won't change. You can view a process tree via Help > Open Process Explorer to help understand this better.

EDIT: Formatting

Re: Speeding up VSCode extensions in 2022

#26

Post by peakaboo » Fri, Jan 28, 2022, 6:59 PM UTC

jjwiseman wrote: post #20

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and samp…

I use treesitter from neovim so it's pretty crazy fast. :)

Re: Speeding up VSCode extensions in 2022

#27

Post by capableweb » Fri, Jan 28, 2022, 7:25 PM UTC

jjwiseman wrote: post #20

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and samp…

It seems to be mentioned, unless I misunderstand what you mean. The second-last section in the article begins with this:

> If you really need to do some CPU intensive work, it’s now possible to offload some of the workload to a language server. This allows you to implement the bulk of your extension in another language (for instance, writing Rust code and compiling it down to WASM).

The section above also mentions tree-sitter.

Re: Speeding up VSCode extensions in 2022

#28

Post by jjwiseman » Fri, Jan 28, 2022, 7:33 PM UTC

capableweb wrote: post #27

jjwiseman wrote: post #20

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and samp…

It seems to be mentioned, unless I misunderstand what you mean. The second-last section in the article begins with this: > If you really need to do some CPU intensive work, it’s now possible to offload some of the workload to a language server. This allows you to implement the bulk of your extension in another language (for instance, writing Rust code and compiling it down to WASM). The section above also mentions tr…

I think you're right. I got confused because he talks about compiling to WASM; I'm not knowledgeable about WASM but a language server is an entirely separate process, which means you can compile your Rust to native x86 or whatever you want. I assume Rust compiled to native machine code is faster than WASM? I guess the one advantage of WASM in that case is that you don't have to compile the language server for every platform the extension runs on. (In my case, I compile the language server for x86 Linux and Windows.)

Re: Speeding up VSCode extensions in 2022

#29

Post by patrec » Fri, Jan 28, 2022, 7:37 PM UTC

jjwiseman wrote: post #20

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and samp…

> A note on my experience with tree-sitter: It's awesome in every way except one.

One more: the Rube-Goldberg build setup. You need rust, npm, node and a C compiler as well as Docker or Emscripten. Tree-sitter is mostly written in Rust and there are Rust bindings, but you can't use them to for the wasm compilation target because it requires linking a C generated grammar which breaks things. I hope they'll move on to something that's possible to build and use with pure Rust.

Re: Speeding up VSCode extensions in 2022

#30

Post by jjwiseman » Fri, Jan 28, 2022, 7:57 PM UTC

jjwiseman wrote: post #20

One idea for speedups that this article doesn't mention, that I've used very successfully, is splitting an extension into two parts: The Javascript extension that runs in VS Code, and a separate language server[1] written in whatever fast language you desire (without having to compile to WASM). I do this with an extension I develop, and it works very well. Microsoft even provides a library to make this easy, and samp…

I forgot to mention one other thing about tree-sitter: the error recovery/tolerance, which is a requirement for practical parsing applications. It does an excellent job of returning a parse tree even if the text contains tons of errors. Instaparse, besides being much slower, just gave up if a single character was out of place. Super rarely there will be an error that causes tree-sitter to fail more comprehensively, but it is one of the few parser frameworks that even tries.