Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust Glancer: Rust LSP using 100x less RAM

rust-glancer.github.io

Post reply on HN
Print view View original

111–120 of 129 posts

Re: Rust Glancer: Rust LSP using 100x less RAM

#111

Post by rtpg »

popzxc wrote: post #35
potamic wrote: post #34

Earlier quoted context omitted.

Could you elaborate a bit on why RA's incremental approach takes more memory? Intuitively it feels that it should take less, because you're only processing what you need? Whereas you seem to indicate that you save a full analysis snapshot to disk and load it all up when needed? Shouldn't that consume the max memory for a workspace? Fantastic project btw, and it couldn't come at a better time. With the way prices are…

It's explained in the blog post, but in short: rust-analyzer stores the data it needs in memory all the time, while Rust Glancer might consume more memory during indexing (because it's not lazy and does more indexing), but after that it only loads _necessary_ information for the duration of the query. Several things here: 1. We don't need all the information (project can have 1000+ dependencies, while query might onl…

Let's say I type "SomeStructure" and haven't imported it, how does rust glancer _not_ end up poking at all the dependencies to figure out where it is, and to import it?

In some sense I feel like that example is the canonical "you need a full picture of the world" case, to the point that it's probably worth special casing _just_ that, and using local analysis for everything... but it seems hard to avoid that pain on any typo for example

Inversely... do you have a ballpark for what the size of the on-disk structures end up being? Not that it matters _that_ much but Rust projects already have a tendancy to be disk hogs during development

Re: Rust Glancer: Rust LSP using 100x less RAM

#112

Post by worik »

hofiflo wrote: post #102
UltraSane wrote: post #60

Earlier quoted context omitted.

"I personally don’t agree with “LLMs are just a tool”" Then what are they?

I’ve observed that people saying “LLMs are just a tool” usually compare them to language server implementations, compilers, and more. I disagree with this view because the tools they’re compared to are usually deterministic in the sense of they’re not just a blackbox that sometimes answers one way, sometimes another depending on whether the API provider changes the model weights, the temperature, etc.

I am part of the "LLMs are tools" crowd

I use nail guns as the analogous tool. Perhaps I know a few carpenters

Pushing back against two things:

* Inappropriate albeit understandable anthropomorphising of the technology

* Misanthropy. The capitalist wet dream of doing away with labour entirely

Re: Rust Glancer: Rust LSP using 100x less RAM

#113

Post by popzxc »

rtpg wrote: post #111
popzxc wrote: post #35

Earlier quoted context omitted.

It's explained in the blog post, but in short: rust-analyzer stores the data it needs in memory all the time, while Rust Glancer might consume more memory during indexing (because it's not lazy and does more indexing), but after that it only loads _necessary_ information for the duration of the query. Several things here: 1. We don't need all the information (project can have 1000+ dependencies, while query might onl…

Let's say I type "SomeStructure" and haven't imported it, how does rust glancer _not_ end up poking at all the dependencies to figure out where it is, and to import it? In some sense I feel like that example is the canonical "you need a full picture of the world" case, to the point that it's probably worth special casing _just_ that, and using local analysis for everything... but it seems hard to avoid that pain on a…

On the first question -- auto imports is an example of a "heavy" functionality since we indeed need to scan more than is imported in the project, which is why I'm still working on this to optimize properly (it's decently fast, but I want it to be faster before I'll enable it, since it's also something we want to see in completions, and completions must be _very_ fast). Still, we don't need "everything": we only need _reachable_ items in _direct_ dependencies; typically project has much less direct dependencies than transitive ones (e.g. 20 direct dependencies can fan out to 1000 transitive deps), and each dependency usually has less exported items than total amount of items. Finally, for that you only need semantic data (e.g. items) rather than bodies. So while it's a big task (though finding references is still significantly bigger and tougher, because that's where it gets to "it can be anywhere in reverse dependencies bodies), it's not "we need the whole world".

As for the size -- the Rust Glancer artifacts for Rust Glancer itself currently take 225mb. And that's proportial to the project size only, e.g. it doesn't grow over time on its own.

Re: Rust Glancer: Rust LSP using 100x less RAM

#114

Post by popzxc »

rdescartes wrote: post #87
popzxc wrote: post #29

Hey! Author here. Happy to answer any questions.

Could you mind to tell me what is your plan on proc-macros ? You said you have an idea to “will not require actual code execution”. On the other hand, RA current method of handling in Proc-Macros are very fragile.

The exact shape is TBD (I target proc macros support for 0.3.0, whereas 0.2.0 will be about completeness/editors support), but in short -- I'm thinking about "plugin"/DSL architecture where proc macro _effects_ can be described beforehand and applied by the LSP without actual code execution. Executing proc macros is a lot of effort, but LSP mostly cares about the externally observable effects, which is a much smaller subset.

Re: Rust Glancer: Rust LSP using 100x less RAM

#116

Post by popzxc »

hacki11 wrote: post #115

Can you tell about how you write/read your data to/from disk? I‘m also exploring the incremental world using an approach like salsa+rocksdb. Similar like https://docs.rs/qbice/latest/qbice/ . Would be great to learn about your approach.

Well, it's hard to give a concise overview, but in short -- I use `wincode` for serde (without zero-copy deserialization though; I've tried zero-copy first with rkyv but it was not trivial at all so I abandoned this idea, plus FS does not dominate the costs so far anyway).

Data is written at the end of indexing phase, during write we hold a lock, and the codebase is aware that offloaded data can be corrupted/outdated (mostly relevant for cache).

Query processing works on top of transaction, and the transaction holds data loaded from files for the duration of the query, and frees once the query is dropped.

There is a ton of tricky parts there, but I guess that's the gist of it.

If you're interested, here are the relevant parts of the source code: https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crate...

Re: Rust Glancer: Rust LSP using 100x less RAM

#117

Post by popzxc »

hacki11 wrote: post #115

Can you tell about how you write/read your data to/from disk? I‘m also exploring the incremental world using an approach like salsa+rocksdb. Similar like https://docs.rs/qbice/latest/qbice/ . Would be great to learn about your approach.

And regarding salsa+rocksdb, actually Alex Kladov advocated exactly for that in the post about Rust Glancer and rust-analyzer architecture: https://matklad.github.io/2026/08/21/rust-glancer.html

Though I believe it's more of a direction for rust-analyzer rather Rust Glancer, at least for now.

Re: Rust Glancer: Rust LSP using 100x less RAM

#118

Post by phplovesong »

juntz wrote: post #74
phplovesong wrote: post #52

Earlier quoted context omitted.

Hostile fork!

Oh I'm sorry for that...I just want to express the appreciation. So if you don't mind, could you tell me why this would looks like some Malicious comments. I'm new for the opensource project, if there any impolite for the "fork",I would like to apologize for everyone here

Its a goddamn joke, as "hostile fork" usully means jack and does not work out. Seems HN is full of either bots, or users are so deep in AI mania that the forgot sarcasm.

Re: Rust Glancer: Rust LSP using 100x less RAM

#119

Post by juntz »

phplovesong wrote: post #118
juntz wrote: post #74

Earlier quoted context omitted.

Oh I'm sorry for that...I just want to express the appreciation. So if you don't mind, could you tell me why this would looks like some Malicious comments. I'm new for the opensource project, if there any impolite for the "fork",I would like to apologize for everyone here

Its a goddamn joke, as "hostile fork" usully means jack and does not work out. Seems HN is full of either bots, or users are so deep in AI mania that the forgot sarcasm.

Got the joke now my 'Forking it!' sounded more hostile than intended. Will read the room better next time.
Post reply on HN