Live data from Hacker News

The Rust compiler is still getting faster

blog.mozilla.org

41–50 of 100 posts

Re: The Rust compiler is still getting faster

#41
post #36
post #30

Earlier quoted context omitted.

> Why doesn’t every language give me a batteries-included digraph ADT? I think such a type would be less useful than you'd think, for precisely the same reason why a linked list in the stdlib is pretty much useless: almost always, you don't want the stdlib allocating nodes; you want an intrusive data structure instead. Really, the problem is that the compiler needs to store extra data with the nodes that the stdlib c…

> you don't want the stdlib allocating nodes; you want an intrusive data structure instead. Don't C++'s std::list and Rust's std::collections::LinkedList effectively implement the moral equivalent (insofar as memory layout is concerned) of an intrusive list, though?

Yes, but they hide the memory layout as an implementation detail, thus negating most advantages of an intrusive list.

You can't call `std::list::erase()` with a Node* , and you can't write a function that converts from Node* to `std::list::iterator` in standard C++, even though it's really just a matter of subtracting a fixed number of bytes from the pointer value. So you instead need to store the `std::list::iterator` in the `Node`, wasting memory.

You can kind of cheat by always using `std::list::iterator` instead of Node* , but that means you can't use normal methods, because the `this` pointer will lack access to the iterator. And you always need to pass around a pair of `std::list&` and iterator, because the iterator alone isn't sufficient to delete the node or insert elements before/after it. Usually the code ends up being a lot simpler if class Node just reimplements a linked list.

And that's still the simple case where each node is only contained in a single data structure at a time, which (at least in compilers) is rarely sufficient.

Re: The Rust compiler is still getting faster

#42
post #15

Faster compiler is nice, but you know what's faster? Not having to compile anything. I'm also looking forward to crates.io serving precompiled crates ( https://www.ncameron.org/blog/cargo-in-2019/ )

I'm a Rust noob but I think skunkopalypse was making some good points. I wish people didn't vote down his/her comment to death and actually replied to it so I could learn why he/she wasn't right.

Re: The Rust compiler is still getting faster

#43
post #11

Earlier quoted context omitted.

Visual C++ is part of those few. Rust is still far from Delphi, Eiffel, .NET Native experience though. Although it is great that it keeps improving.

Is there a blog post about Visual C++ being multi-threaded end-to-end ? (e.g. doing parsing, type-checking, etc. of a single TU in parallel using all cores ?) That would be super interesting to read because it is mainly a C++ compiler and some C++ features like two-phase lookup and macros make it quite hard to do things in parallel. You have to do things in a certain order but I suppose that if its query-based as wel…

As the documentation says: "The /MP (Build with Multiple Processes) compiler flag specifies the number of cl.exe processes that simultaneously compile the source files. The /cgthreads option specifies the number of threads used by each cl.exe process." I've never used /CGTHREADS but /MP is working well (except for the precompliled header file).

Re: The Rust compiler is still getting faster

#44

Earlier quoted context omitted.

> - CMake > - Easy Pick one. I'm sure there is though. It wouldn't be hard with make but then again I'm much more familiar with it than cmake.

So I manage to get it to work in the command line using `CXXFLAGS="$CXXFLAGS -fsyntax-check" cmake ... && make` !

One issue with that is that is, unless there is magic handling of the flag in cmake, it might still attempt (and fail) to run the linking stages. You might need custom targets jut for this.

Re: The Rust compiler is still getting faster

#45
post #11

Earlier quoted context omitted.

Visual C++ is part of those few. Rust is still far from Delphi, Eiffel, .NET Native experience though. Although it is great that it keeps improving.

Is there a blog post about Visual C++ being multi-threaded end-to-end ? (e.g. doing parsing, type-checking, etc. of a single TU in parallel using all cores ?) That would be super interesting to read because it is mainly a C++ compiler and some C++ features like two-phase lookup and macros make it quite hard to do things in parallel. You have to do things in a certain order but I suppose that if its query-based as wel…

Visual C++ MSDN blog has a couple of blog entries how they refactored the compiler to use AST, added incremental compilation and incremental linking.

But as far as I remember it isn't fully multi-threaded across all phases.

Re: The Rust compiler is still getting faster

#46
post #11

Earlier quoted context omitted.

Visual C++ is part of those few. Rust is still far from Delphi, Eiffel, .NET Native experience though. Although it is great that it keeps improving.

In my experience VC++ is order of magnitude slower than GCC or clang though. Might have been a bad build system though.

Only when incremental compiler, incremental linking and pre-compiled headers are disabled.

Re: The Rust compiler is still getting faster

#47
post #18

Earlier quoted context omitted.

The one thing I’ve always wanted from a language runtime is for the data structures used by the compiler to be exposed in the stdlib. E.g. every compiler uses control flow graphs; so why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? It’d be a great boon to writing you…

> why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? Rust does give you access to its internal data-structures on nightly. These change quite often, so you will need to update code that uses them pretty much every week. Why doesn't many language does this in some stabl…

Java and .NET do in some form.

Then you have Eiffel, Smalltalk and Lisp variants.

Re: The Rust compiler is still getting faster

#48
post #39

Earlier quoted context omitted.

> why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? Rust does give you access to its internal data-structures on nightly. These change quite often, so you will need to update code that uses them pretty much every week. Why doesn't many language does this in some stabl…

I want to make a distinction between three kinds of data structures. 1. Implementation-specific thing that has no formal equivalent. Structs of structs of structs, with business logic intermingled with the structure. Plenty of examples of this. I’m not suggesting anyone put these in the stdlib; that’d be silly. 2. ADT with no formalism behind it, that implements a particular set of behaviours “for best performance”,…

> It’s #3 that I would suggest is a good candidate for stdlib inclusion

You started arguing that the Rust compiler should expose its internal data-structures, which it does, and somehow ended arguing that the Rust standard library should expose spatial data-structures for geometry processing.

I have no idea how you got from one to the other, but writing a huge wall of text full of disorganized thoughts shows very little appreciation for the time of those participating in this discussion thread, so I won't interact with you anymore.

Re: The Rust compiler is still getting faster

#49
post #47

Earlier quoted context omitted.

> why doesn’t every language give me a batteries-included digraph ADT, that has every feature you’d need to implement a control flow graph, such that the compiler’s CFG is just a stdlib digraph instance? Rust does give you access to its internal data-structures on nightly. These change quite often, so you will need to update code that uses them pretty much every week. Why doesn't many language does this in some stabl…

Java and .NET do in some form. Then you have Eiffel, Smalltalk and Lisp variants.

Rust does expose _all_ of its compiler internal data-structures though, not just some subset of it.

Scheme and Lisp, for example, only expose their AST. If you only care about the AST, you can access it from stable Rust, and there are great libraries for working with it, doing AST folds, semi-quoting, etc.

The OP wanted to work on the CFG graph. You could write a library to compute the CFG from the AST, but you don't have to because Rust exposes this as well. The CFG data-structures are much more tied to the intermediate representations of the Rust compiler, and these do change over time as new features are added to the language.

Some tools do use the compiler internal CFGs though. For example, rust-clippy is a linter that's built on top of most of the compiler-internal data-structures, not only type-checking, but also CFGs. The rust-semverver is a tool that detects semver breaking changes between the last released version of a library and the current one, is built on top of the type checking data-structures, and can deal with all kinds of generics (types, lifetimes, etc.).

These tools are typically tied to particular versions of the nightly compiler and do break often, but for example rust-clippy is distributed via rustup, so you always get a version that works with whatever nightly compiler you have, and well, you also get a version that "magically" works with a stable Rust compiler.

Other people have built all sort of tools on top of this, from Rust interpreters, to instrumentation passes that compute the maximum stack size requirement of embedded applications, e.g., by using the CFG to compute the deepest possible stack frame of a program, and the size of the stack frame for that case.

Re: The Rust compiler is still getting faster

#50
post #43

Earlier quoted context omitted.

Is there a blog post about Visual C++ being multi-threaded end-to-end ? (e.g. doing parsing, type-checking, etc. of a single TU in parallel using all cores ?) That would be super interesting to read because it is mainly a C++ compiler and some C++ features like two-phase lookup and macros make it quite hard to do things in parallel. You have to do things in a certain order but I suppose that if its query-based as wel…

As the documentation says: "The /MP (Build with Multiple Processes) compiler flag specifies the number of cl.exe processes that simultaneously compile the source files. The /cgthreads option specifies the number of threads used by each cl.exe process." I've never used /CGTHREADS but /MP is working well (except for the precompliled header file).

That's for different source files though, right? Try having a 10Mb auto-generated C++ file, see how fast it compiles, regardless how powerful your computer is.
Post reply on HN