Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What part of Rust compilation is the bottleneck?

kobzol.github.io

Post reply on HN
Print view View original

11–20 of 62 posts

Re: What part of Rust compilation is the bottleneck?

#11

Post by KingOfCoders »

makapuf wrote: post #9
KingOfCoders wrote: post #7

One reason for me moving from Rust to Go was compilation speed. Go is a simpler language, so apples to oranges, but Go compiles so fast, which to me makes development very different.

That is definitely true, but from the article it's the backend that takes time, not the frontend where the language itself resides. If you compile go from llvm, it maybe as long as rust.

I wonder why we do not split up compilation more - especially for web developer. Rust does this a little with "check", C with "-O".

I want fast compilation for my dev cycle or for unit tests, I want slow compilation with optimizations, escape analysis, correctness etc. for production (the distinction between a compiler and linter is also not clear, some compiles do what linters do in other languages).

Re: What part of Rust compilation is the bottleneck?

#12

Post by jokethrowaway »

KingOfCoders wrote: post #11
makapuf wrote: post #9

Earlier quoted context omitted.

That is definitely true, but from the article it's the backend that takes time, not the frontend where the language itself resides. If you compile go from llvm, it maybe as long as rust.

I wonder why we do not split up compilation more - especially for web developer. Rust does this a little with "check", C with "-O". I want fast compilation for my dev cycle or for unit tests, I want slow compilation with optimizations, escape analysis, correctness etc. for production (the distinction between a compiler and linter is also not clear, some compiles do what linters do in other languages).

We do, you have profile configuration for dev or for release

You can get pretty big differences in terms of compile speed / binary size

Re: What part of Rust compilation is the bottleneck?

#13

Post by jokethrowaway »

Over the years compile speed improved quite a bit (recently this: https://blog.rust-lang.org/2023/11/09/parallel-rustc.html made quite the difference)

If we can squeeze more performance that's great but the largest concern I have around compilation is with the size of the target directory

It can balloon up to node_modules levels

Re: What part of Rust compilation is the bottleneck?

#14

Post by hobs »

foota wrote: post #4
hobs wrote: post #2

Good article, just throwing out there that flamegraphs would be exactly what you need for visualizing this stuff.

It looks like the profile they're build on already supports those, I think the intention here is to present a sort of at a glance view that could be quickly analyzed.

Makes sense. In my opinion I would just generate fake flamegraphs then - they are much more readable and the format is dummy easy to fake.

Re: What part of Rust compilation is the bottleneck?

#15

Post by GrumpySloth »

makapuf wrote: post #9
KingOfCoders wrote: post #7

One reason for me moving from Rust to Go was compilation speed. Go is a simpler language, so apples to oranges, but Go compiles so fast, which to me makes development very different.

That is definitely true, but from the article it's the backend that takes time, not the frontend where the language itself resides. If you compile go from llvm, it maybe as long as rust.

It’s not just Rust. Practically every compiler based on LLVM is slow. Swift, Zig, Clang.

The Go compiler being written from scratch based on the Plan9 C compiler is a huge advantage.

Re: What part of Rust compilation is the bottleneck?

#16

Post by davidhyde »

makapuf wrote: post #10
davidhyde wrote: post #6

If you make a small change to your application, the Rust compiler does a significant amount of rework. That is, it recompiles a lot of code that it has already compiled before. There are valid technical reasons for this because of how LLVM works or that the linker needs to rewrite all addresses. Yes, incremental compilation is a thing but it’s too coarse IMO. To me it seems that taking an extremely fine grained appro…

Depends on your target; if you have tiny compilation units you won't be able to optimize /inline on a broad target, that's why single unit compilation is an option (that may or may not improve the resut)

This is true but what is really happening is that the frontend and backend cannot communicate intent effectively because of they way they are separated. The frontend doesn’t know what is important for optimisation because that’s not its job and the backend only sees the code the front gives it (never a wholistic view). So the easiest (and slowest) approach is to do everything over and over again.

Increasing Codegen units (multi unit compilation) is just the user taking a risk that splitting things up will not affect performance optimisations. Nothing smart about it.

If you had tiny compilation units and the frontend understood their significance to the backend then it would be able to build a graph of dirty code to be recompiled when a small piece of it changes.

Re: What part of Rust compilation is the bottleneck?

#17

Post by anonyfox »

KingOfCoders wrote: post #7

One reason for me moving from Rust to Go was compilation speed. Go is a simpler language, so apples to oranges, but Go compiles so fast, which to me makes development very different.

second that. also another point: I write most Go code using only the stdlib, so there is no dependency web to take care on top of the actual code.

Re: What part of Rust compilation is the bottleneck?

#18

Post by yxhuvud »

makapuf wrote: post #10
davidhyde wrote: post #6

If you make a small change to your application, the Rust compiler does a significant amount of rework. That is, it recompiles a lot of code that it has already compiled before. There are valid technical reasons for this because of how LLVM works or that the linker needs to rewrite all addresses. Yes, incremental compilation is a thing but it’s too coarse IMO. To me it seems that taking an extremely fine grained appro…

Depends on your target; if you have tiny compilation units you won't be able to optimize /inline on a broad target, that's why single unit compilation is an option (that may or may not improve the resut)

One problem they really need to address though is that as soon as you put everything in a single compilation unit, then compilation is single threaded and dog slow. Compilation units maybe make sense for C and C++, but for other languages they are just a way to structure the compilation. It should be automatic, and just better all around.

Re: What part of Rust compilation is the bottleneck?

#19

Post by kobzol »

jokethrowaway wrote: post #13

Over the years compile speed improved quite a bit (recently this: https://blog.rust-lang.org/2023/11/09/parallel-rustc.html made quite the difference) If we can squeeze more performance that's great but the largest concern I have around compilation is with the size of the target directory It can balloon up to node_modules levels

There have been some recent improvements to this, but yeah, it can be still quite large. There is a WIP development of a garbage collector in Cargo that could help with this.

Re: What part of Rust compilation is the bottleneck?

#20

Post by kobzol »

hobs wrote: post #14
foota wrote: post #4

Earlier quoted context omitted.

It looks like the profile they're build on already supports those, I think the intention here is to present a sort of at a glance view that could be quickly analyzed.

Makes sense. In my opinion I would just generate fake flamegraphs then - they are much more readable and the format is dummy easy to fake.

Yeah, I actually generated these small charts out of a flamegraph, because it contains too much information and isn't easily split into three distinct parts. And once you condense the information into just 3 blocks, then using a flamegraph doesn't really add any further value, IMO.
Post reply on HN