Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Arborium: Tree-sitter code highlighting with Native and WASM targets

arborium.bearcove.eu

Post reply on HN
Print view View original

31–40 of 48 posts

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#31

Post by debugnik »

oersted wrote: post #17
GolDDranks wrote: post #14

Earlier quoted context omitted.

It's a Rust library (comprised of a bunch of crates) that wraps a high-performance, high-accuracy syntax highlighter (called Tree-sitter) with vetted support for almost 100 programming/markup languages. You can use it as a normal Rust library, or you can use the JavaScript/WASM wrapper to highlight source code on a web page.

> high-accuracy syntax highlighter (called Tree-sitter) Just wanted to note that tree-sitter is lower-level and more general: it's an incremental parser that is specialised for gracefully and efficiently parsing partially-correct code snippets or code being edited live. It's an important building block of the highlighter, but it needs more on top to complete the package. It can be used for anything that requires awar…

If only it were usable for really-correct parsing. In my experience error recovery is so aggressive it will accept broken ASTs without marking any node as an error. Plus, you can't really solve some ambiguities without C-based lexer hacks.

I wonder if targeting the Tree-sitter ABI directly could be a viable way to write more accurate parsers in an actual programming language while piggybacking on the ecosystem. Could tree-sitter's runtime ABI be adapted for GLL parsers instead of GLR? I haven't looked deep into it yet.

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#32

Post by gorjusborg »

GolDDranks wrote: post #14
teo_zero wrote: post #4

Sorry, but I can't understand what this actually is. A library, a stand-alone tool, a Rust crate? What users does it target? Text editors, website creators?

It's a Rust library (comprised of a bunch of crates) that wraps a high-performance, high-accuracy syntax highlighter (called Tree-sitter) with vetted support for almost 100 programming/markup languages. You can use it as a normal Rust library, or you can use the JavaScript/WASM wrapper to highlight source code on a web page.

Is the 'regex hater club' subtitle related to using regex to 'parse' rather than using something like tree-sitter that actually parses?

I also had a hard time understanding the context given just the link.

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#33

Post by knallfrosch »

mg wrote: post #29
debazel wrote: post #21

Earlier quoted context omitted.

The example on their website is editable and it looks like they overlay the highlighted output on top of the textarea with `pointer-events: none` like you mentioned. The code isn't minified so you can see how they do it by looking at the `doHighlight()` function here https://arborium.bearcove.eu/pkg/app.generated.js

Oh, you are right! Hmm .. and the approach already shows its weaknesses when I play with it: When I search for something on the page, it gives me twice as many hits as there are. And jumps around two times to each hit when I use the "next" button. I wonder if that is fixable.

One simply needs the Highlight API. I held back, but now even Firefox ESR supports it.

https://developer.mozilla.org/en-US/docs/Web/API/Highlight

All the trickery vanishes and you get first-class CSS support.

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#34

Post by conartist6 »

debugnik wrote: post #31
oersted wrote: post #17

Earlier quoted context omitted.

> high-accuracy syntax highlighter (called Tree-sitter) Just wanted to note that tree-sitter is lower-level and more general: it's an incremental parser that is specialised for gracefully and efficiently parsing partially-correct code snippets or code being edited live. It's an important building block of the highlighter, but it needs more on top to complete the package. It can be used for anything that requires awar…

If only it were usable for really-correct parsing. In my experience error recovery is so aggressive it will accept broken ASTs without marking any node as an error. Plus, you can't really solve some ambiguities without C-based lexer hacks. I wonder if targeting the Tree-sitter ABI directly could be a viable way to write more accurate parsers in an actual programming language while piggybacking on the ecosystem. Could…

Now you're in my wheelhouse! Come check out https://github.com/bablr-lang/. I'm gearing up for a big release announcement here once I fix these bugs and ship all the latest code.

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#35

Post by catapart »

Daaang! On the one hand, as someone developing a code-example custom element[0] that includes highlighting, this is kind of a "so close, but not quite" situation where I really wish this was something I could use for that, but it's probably too heavy to ship around for something so "simple" (as far as users' default expectation; not implying that highlighting text is simple).

But then, on the other hand, I had given up on a scratch code editor for a game development project I'm working on, and just loosely wrapped up the monaco editor which I'm guessing is going to be pretty bare when I actually get around to trying to code with it, in browser (I'm aware that it is robust, but from what I gather, a lot of its features come from third-party dev as a way to keep the core functionality pure). Given that I want to be able to script in C# (aside from just js/ts), I was sure I was going to have to figure something complicated out.

But, honestly, I think this solves all of my most concerning issues! What a sweet little library!

[0] https://magnitce-code-example-e81613.gitlab.io/ (please excuse the unfinished-ness; I'm working on a JSDoc-to-documentation library that automates the documentation for me so there are minor issues, like the install text not changing on selection)

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#36

Post by debazel »

mg wrote: post #29
debazel wrote: post #21

Earlier quoted context omitted.

The example on their website is editable and it looks like they overlay the highlighted output on top of the textarea with `pointer-events: none` like you mentioned. The code isn't minified so you can see how they do it by looking at the `doHighlight()` function here https://arborium.bearcove.eu/pkg/app.generated.js

Oh, you are right! Hmm .. and the approach already shows its weaknesses when I play with it: When I search for something on the page, it gives me twice as many hits as there are. And jumps around two times to each hit when I use the "next" button. I wonder if that is fixable.

There is a neat `inert` html attribute you can use to disable all interactions as well as hide the text from ctrl+f searches. (Sadly Safari is the weird one out, and does not exclude the content from searches.)

https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#37

Post by brabel »

falcor84 wrote: post #23
solarkraft wrote: post #20

Earlier quoted context omitted.

I agree! Not only do they write engaging blog posts, they also produce engaging videos! https://youtube.com/@fasterthanlime

I suppose this isn't the case here, but found it funny how these 3 successive positive comments read almost exactly like those bot comment chains that I see on other platforms trying to pull people into crypto scams.

Haha, I confess I did sound like a bot... but sorry to disappoint, I am human!

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#39

Post by ComputerGuru »

Completely appalled to learn that docs.rs lets you inject any html/css/js you want into the live site (on pages documenting your crate). I love the flexibility but shudder at the security hole the size of, oh, I don’t know, the Grand Canyon.

It’s not a new discovery, I just didn’t know docs.rs (intentionally) wasn’t blocking this. Cf https://docs.rs/pwnies/0.0.13/pwnies/

(This all makes more sense if you read the blog post instead of the direct arborium link: https://fasterthanli.me/articles/my-gift-to-the-rust-docs-te...)

Re: Arborium: Tree-sitter code highlighting with Native and WASM targets

#40

Post by psychoslave »

brabel wrote: post #37
falcor84 wrote: post #23

Earlier quoted context omitted.

I suppose this isn't the case here, but found it funny how these 3 successive positive comments read almost exactly like those bot comment chains that I see on other platforms trying to pull people into crypto scams.

Haha, I confess I did sound like a bot... but sorry to disappoint, I am human!

That's what a bot would typically say.

On this side, I'm not a bot, and fortunately even in 2025 nobody on the internet knows that you are a dog.

Post reply on HN