Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rust compiler is still getting faster

blog.mozilla.org

Post reply on HN
Print view View original

21–30 of 100 posts

Re: The Rust compiler is still getting faster

#21

Post by fluffything »

ynik wrote: post #20
fluffything wrote: post #5

Earlier quoted context omitted.

I just use `cargo watch -x test -d 5` to run all my tests after I pause modifying code for 5 seconds. My editor (emacs) uses `cargo watch -x check -d 0.5` to run `cargo check` (which is blazing fast for incremental edits) to type check all the code and show "red squiggles" with the error messages inline. So my interactive workflow with Rust is only edit-"type check"-edit-"type check" where "type check" takes often le…

> In C++ there was no way to only run type checking gcc has -fsyntax-only. Despite the option name, this also includes type checking and template instantiation. AFAIK it reports all compiler errors, though it skips some warnings that are computed by the optimizer (e.g. -Wuninitialized).

Is there an easy way to tell CMake or Makefiles to use it ?

I never invoke clang or gcc directly. When using cargo, I use `cargo check` instead of `cargo build`. But in C or C++ depending on the project `make check` might not exist, or it might build all tests and run them, or do something else entirely like checking the formatting using clang-format.

Re: The Rust compiler is still getting faster

#22

Post by moomin »

derefr wrote: post #18
nindalf wrote: post #7

Earlier quoted context omitted.

The rust compiler team continues to set ambitious goals. Last I checked they wanted to extract much of the compiler front end into independent crates that can be reused by the Language Server. That's a pretty daunting refactoring, but the way they're going seems like it's achievable.

The one thing I’ve always wanted from a language runtime is for the data structures used by the compiler to be exposed in the stdlib. E.g. every compiler uses control flow graphs; so why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? It’d be a great boon to writing you…

The thing is a) these structures change all the time and b) published APIs shouldn't change all the time. Which is why Erlang uses a public model and a private model and why most don't bother.

Doesn't mean I wouldn't like it myself, but it is a genuine drag on continued compiler development.

Re: The Rust compiler is still getting faster

#23

Post by fluffything »

pjmlp wrote: post #11
fluffything wrote: post #3

The rust compiler now has features that few or none C and C++ compilers have: incremental compilation within a single translation unit, pipelined compilation, multi-threaded and lazy query-based compilation, ... Implementing each of these features have required whole program refactorings in a large-scale codebase performed by few individuals while hundreds of other developers where simultaneously evolving the softwar…

Visual C++ is part of those few. Rust is still far from Delphi, Eiffel, .NET Native experience though. Although it is great that it keeps improving.

Is there a blog post about Visual C++ being multi-threaded end-to-end ? (e.g. doing parsing, type-checking, etc. of a single TU in parallel using all cores ?)

That would be super interesting to read because it is mainly a C++ compiler and some C++ features like two-phase lookup and macros make it quite hard to do things in parallel. You have to do things in a certain order but I suppose that if its query-based as well it will work.

The only option I know is /CGTHREADS but that only uses multiple-threads for optimizations and code generation which is something that the Rust compiler has been able to do for a very long time (e.g. there these are called codegen-units and LLVM supports these so it is quite trivial for a frontend to do so as well).

Re: The Rust compiler is still getting faster

#24

Post by ChrisSD »

derefr wrote: post #18
nindalf wrote: post #7

Earlier quoted context omitted.

The rust compiler team continues to set ambitious goals. Last I checked they wanted to extract much of the compiler front end into independent crates that can be reused by the Language Server. That's a pretty daunting refactoring, but the way they're going seems like it's achievable.

The one thing I’ve always wanted from a language runtime is for the data structures used by the compiler to be exposed in the stdlib. E.g. every compiler uses control flow graphs; so why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? It’d be a great boon to writing you…

Surely the risk there is exposing implementation details in the standard library? An separate library that's shared with the compiler may make sense, but going into the stdlib tends to come with a promise of stability.

Re: The Rust compiler is still getting faster

#25

Post by GrayShade »

rehemiau wrote: post #8
howenterprisey wrote: post #6

Does LLVM still take up much of the overall time spent by the Rust compiler? I was thinking of getting involved over there as the most effective way to make speed-ups happen.

Probably yes, the LLVM linker is really slow

Rust is only using LLD on ARM targets.

Re: The Rust compiler is still getting faster

#26

Post by skunkpocalypse »

pornel wrote: post #15

Faster compiler is nice, but you know what's faster? Not having to compile anything. I'm also looking forward to crates.io serving precompiled crates ( https://www.ncameron.org/blog/cargo-in-2019/ )

Ew, gross.

Crates.io is already a security trainwreck in progress. Do we really need to add even more attack vectors?

It's also solving a non-problem. I modify source code downloaded from crates.io zero times per day, so I compile each crate only once. Compile times matter for code I write myself: I modify (and therefore compile) that code dozens of times per day.

Re: The Rust compiler is still getting faster

#27

Post by gpderetta »

fluffything wrote: post #5
sddfd wrote: post #2

This is great! Many dev hours are spend waiting for the compiler, every second counts! The second order effects are even worse. After a minute, the programmer will start thinking about other things, running flow. If compiles regularly take 5min, devs will leave their desks (and honestly, who can blame them for it).

I just use `cargo watch -x test -d 5` to run all my tests after I pause modifying code for 5 seconds. My editor (emacs) uses `cargo watch -x check -d 0.5` to run `cargo check` (which is blazing fast for incremental edits) to type check all the code and show "red squiggles" with the error messages inline. So my interactive workflow with Rust is only edit-"type check"-edit-"type check" where "type check" takes often le…

> In C++ there was no way to only run type checking

-fsyntax-check on GCC and IIRC clang as well.

If you use Emacs you can integrate it w/ flycheck.

Re: The Rust compiler is still getting faster

#28

Post by gpderetta »

pjmlp wrote: post #11
fluffything wrote: post #3

The rust compiler now has features that few or none C and C++ compilers have: incremental compilation within a single translation unit, pipelined compilation, multi-threaded and lazy query-based compilation, ... Implementing each of these features have required whole program refactorings in a large-scale codebase performed by few individuals while hundreds of other developers where simultaneously evolving the softwar…

Visual C++ is part of those few. Rust is still far from Delphi, Eiffel, .NET Native experience though. Although it is great that it keeps improving.

In my experience VC++ is order of magnitude slower than GCC or clang though. Might have been a bad build system though.

Re: The Rust compiler is still getting faster

#29

Post by gpderetta »

fluffything wrote: post #21
ynik wrote: post #20

Earlier quoted context omitted.

> In C++ there was no way to only run type checking gcc has -fsyntax-only. Despite the option name, this also includes type checking and template instantiation. AFAIK it reports all compiler errors, though it skips some warnings that are computed by the optimizer (e.g. -Wuninitialized).

Is there an easy way to tell CMake or Makefiles to use it ? I never invoke clang or gcc directly. When using cargo, I use `cargo check` instead of `cargo build`. But in C or C++ depending on the project `make check` might not exist, or it might build all tests and run them, or do something else entirely like checking the formatting using clang-format.

> - CMake > - Easy

Pick one. I'm sure there is though.

It wouldn't be hard with make but then again I'm much more familiar with it than cmake.

Re: The Rust compiler is still getting faster

#30

Post by ynik »

derefr wrote: post #18
nindalf wrote: post #7

Earlier quoted context omitted.

The rust compiler team continues to set ambitious goals. Last I checked they wanted to extract much of the compiler front end into independent crates that can be reused by the Language Server. That's a pretty daunting refactoring, but the way they're going seems like it's achievable.

The one thing I’ve always wanted from a language runtime is for the data structures used by the compiler to be exposed in the stdlib. E.g. every compiler uses control flow graphs; so why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? It’d be a great boon to writing you…

> Why doesn’t every language give me a batteries-included digraph ADT?

I think such a type would be less useful than you'd think, for precisely the same reason why a linked list in the stdlib is pretty much useless: almost always, you don't want the stdlib allocating nodes; you want an intrusive data structure instead. Really, the problem is that the compiler needs to store extra data with the nodes that the stdlib can't know about, but you don't want two separate types `stdlib::GraphNode` and `compiler::ControlFlowNode` because you usually need to be able to convert between these types in both directions. (one direction can be handled by embedding one of the types in the other; but the reverse direction will require overhead for an extra pointer, or horribly unsafe pointer arithmetic)

Of course in Rust, there could still be a digraph trait and the stdlib could still provide generic algorithms.

Though it's also not so rare in compilers that nodes are members of multiple graphs simultaneously (with different edges in each, e.g. control flow nodes are typically not just part of the control flow graph, but also belong to a dominator tree). It's non-trivial to create a graph abstraction that can handle all these cases while remaining efficient (you don't want to put nodes in a HashSet just to check whether a graph algorithm already visited them), so it's not surprising that compiler developers don't bother and just write the algorithm directly for their particular data structures. In the end, most graph algorithms are only about a dozen lines, much simpler than the abstractions that would be required to re-use them across completely different graphs.

Post reply on HN