Optimizations in Syntax Highlighting
code.visualstudio.com
Optimizations in Syntax Highlighting
1–10 of 55 posts
Re: Optimizations in Syntax Highlighting
#2> there is no feasible way to interpret TextMate grammars in the browser even today
That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
Re: Optimizations in Syntax Highlighting
#3> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
Yeah I'm not totally sure what was meant by that. They're plain text, thus parsable.
Maybe they meant that browsers don't usually have access to the file system, but that's changing and also not applicable since they're using Electron and have NodeJS at their disposal.
Re: Optimizations in Syntax Highlighting
#4> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
The reason is that they basically rely on using the Oniguruma regret engine, and reimplementing that in JS would be hella slow.
Re: Optimizations in Syntax Highlighting
#5Shameless plug: my implementation of Sublime's syntax highlighting engine in Rust has similar optimizations and more. I'm not at my computer to benchmark on the same files but it should be >2x as fast as their "after" numbers just based on lines/second for JS-like files.
This evening I'm even trying to port it to a pure Rust regex engine that should eliminate non-Rust code and make it substantially faster.
It also implements the sublime-syntax format which is a superset of tmlanguage that allows even nicer highlighting.
Re: Optimizations in Syntax Highlighting
#6> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
Yeah I'm not totally sure what was meant by that. They're plain text, thus parsable. Maybe they meant that browsers don't usually have access to the file system, but that's changing and also not applicable since they're using Electron and have NodeJS at their disposal.
plain text doesn't imply parsable; natural languages, for instance.
Re: Optimizations in Syntax Highlighting
#7> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
- all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C.
- the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs) in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript
- in the Monaco Editor, we are constrained to a browser environment where we cannot do anything similar
- we have experimented with Emscripten to compile the C library to asm.js, but performance was very poor even in Firefox (10x slower) and extremely poor in Chrome (100x slower).
- we can revisit this once WebAssembly gets traction in the major browsers, but we will still need to consider the browser matrix we support. i.e. if we support IE11 and only Edge will add WebAssembly support, what will the experience be in IE11, etc.
Re: Optimizations in Syntax Highlighting
#8Great article, specially regarding the type of optimisations that were applied.
Re: Optimizations in Syntax Highlighting
#9> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.
The reason is that they basically rely on using the Oniguruma regret engine, and reimplementing that in JS would be hella slow.
> using the Oniguruma regret engine
I'm surprised I've never seen that typo for regex before. It's wonderful.
Re: Optimizations in Syntax Highlighting
#10Earlier quoted context omitted.
The reason is that they basically rely on using the Oniguruma regret engine, and reimplementing that in JS would be hella slow.
> using the Oniguruma regret engine I'm surprised I've never seen that typo for regex before. It's wonderful.
Sometimes you feel a regret. "I know!" you say, "I'll fix things with regretular expressions." Now you have two regrets.