Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizations in Syntax Highlighting

code.visualstudio.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Optimizations in Syntax Highlighting

#11

Post by alimbada »

alexdima wrote: post #7
mrgalaxy wrote: post #2

> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.

- all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C. - the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs) in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript - in the Monaco Editor, we are…

Sorry if I'm missing something, but why do you care about supporting other browsers? Isn't VSCode built on Electron which is a self contained server/browser environment?

Re: Optimizations in Syntax Highlighting

#12

Post by eduren »

alimbada wrote: post #11
alexdima wrote: post #7

Earlier quoted context omitted.

- all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C. - the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs) in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript - in the Monaco Editor, we are…

Sorry if I'm missing something, but why do you care about supporting other browsers? Isn't VSCode built on Electron which is a self contained server/browser environment?

Because the base Monaco editor is used in other products besides Code. The example I've seen thrown around is configuration editing in Azure consoles.

Re: Optimizations in Syntax Highlighting

#13

Post by eduren »

So with the comparisons at the end of the article, does this mean that there were a lot of edge cases where the theming wasn't being correctly applied prior to 1.9? Were there themes that incorporated less stylistic choices because of the limitations?

Re: Optimizations in Syntax Highlighting

#14

Post by wolfgang42 »

alimbada wrote: post #11
alexdima wrote: post #7

Earlier quoted context omitted.

- all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C. - the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs) in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript - in the Monaco Editor, we are…

Sorry if I'm missing something, but why do you care about supporting other browsers? Isn't VSCode built on Electron which is a self contained server/browser environment?

Monaco is a standalone component which also works in the browser: https://microsoft.github.io/monaco-editor/

The article says "It was shipped in the form of the Monaco Editor in various Microsoft projects, including Internet Explorer's F12 tools"; presumably some of those projects also embed it into webpages.

Re: Optimizations in Syntax Highlighting

#15

Post by alexdima »

eduren wrote: post #13

So with the comparisons at the end of the article, does this mean that there were a lot of edge cases where the theming wasn't being correctly applied prior to 1.9? Were there themes that incorporated less stylistic choices because of the limitations?

Yes, those comparisons at the end show differences in rendering caused by the "approximations" used prior to VS Code 1.9. They were all caused by the difference between the ranking rules of CSS selectors and the ranking rules of TM scope selectors

Re: Optimizations in Syntax Highlighting

#16

Post by nojvek »

I once dabbled inside vscode tokenizer code. There is a lot going but putting tokens in a buffer took me by a surprise. It was a very smart implementation.

I think it's their focus on performance and good architecture that makes vscode stand out.

Sure sublime also has great engineering behind it, but being able to contribute and look under the hood of tools, we developers use is very exciting. It feels like a very democratic process.

I remember filing the mini-map bug in Monaco, I'm so glad to see they are working on implementing it in a performant way even though it will require large rewrites of their editor rendering code.

Re: Optimizations in Syntax Highlighting

#17

Post by octref »

trishume wrote: post #5

Shameless plug: my implementation of Sublime's syntax highlighting engine in Rust has similar optimizations and more. I'm not at my computer to benchmark on the same files but it should be >2x as fast as their "after" numbers just based on lines/second for JS-like files. This evening I'm even trying to port it to a pure Rust regex engine that should eliminate non-Rust code and make it substantially faster. It also im…

Sorry but I'm not familiar with Rust -- is it possible to run your lib in all platforms in a Node environment?

And, do you have an issue for the regex port? Would love to see the benchmark.

Re: Optimizations in Syntax Highlighting

#18

Post by gcp »

alexdima wrote: post #7
mrgalaxy wrote: post #2

> there is no feasible way to interpret TextMate grammars in the browser even today That doesn't sound right... but then again I don't know enough about TextMate grammars to argue.

- all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C. - the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs) in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript - in the Monaco Editor, we are…

Maybe slightly offtopic, but is there any chance we could see a VSCode based on Edge/Chakra? I prefer your font rendering over your competitions', and that's a pretty big issue for an editor.

Re: Optimizations in Syntax Highlighting

#20

Post by nickpeterson »

WildUtah wrote: post #10
algorithmsRcool wrote: post #9

Earlier quoted context omitted.

> using the Oniguruma regret engine I'm surprised I've never seen that typo for regex before. It's wonderful.

Sometimes you feel a regret. "I know!" you say, "I'll fix things with regretular expressions." Now you have two regrets.

Maybe "I'll fix this with expressions of regret."
Post reply on HN