Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rust compiler is still getting faster

blog.mozilla.org

Post reply on HN
Print view View original

91–100 of 100 posts

Re: The Rust compiler is still getting faster

#91

Post by gpderetta »

derefr wrote: post #37
ynik wrote: post #30

Earlier quoted context omitted.

> Why doesn’t every language give me a batteries-included digraph ADT? I think such a type would be less useful than you'd think, for precisely the same reason why a linked list in the stdlib is pretty much useless: almost always, you don't want the stdlib allocating nodes; you want an intrusive data structure instead. Really, the problem is that the compiler needs to store extra data with the nodes that the stdlib c…

> In the end, most graph algorithms are only about a dozen lines, much simpler than the abstractions that would be required to re-use them across completely different graphs I guess that depends on the language; in dynamic, pure-functional languages (like Erlang) there’s nowhere you can go for further efficiency beyond “digraph expressed as labelled V and E sets held in dictionaries”, so the data structures themselve…

Yet Boos.Graph (a c++ library) is a generic library of graph algorithms that, in the spirit of the STL, abstracts away the graph representation. Whether the graph is represented via direct pointers or adjacency matrices (however represed) they can be adapted to work very efficiently with the library.

Re: The Rust compiler is still getting faster

#92

Post by commit »

carlmr wrote: post #82
nn3 wrote: post #81

Earlier quoted context omitted.

Yes that's true. Coverity and similar are even slower. But then rust errors out on a lot of things the C++ analyzer wouldn't, a lot of those things are actually being reasonable code. For example I discovered recently that you cannot swap two pointer variables in rust without using unsafe. Surely there is some deep reason for that, that is really important until they can figure out how to fix it in some future Rust v…

>For example I discovered recently that you cannot swap two pointer variables in rust without using unsafe. Surely there is some deep reason for that, that is really important until they can figure out how to fix it in some future Rust version, which they undoubtedly will. Wouldn't this work? https://doc.rust-lang.org/std/mem/fn.swap.html >But then i see that there is actually some progress in Rust to relax some of t…

>Wouldn't this work? https://doc.rust-lang.org/std/mem/fn.swap.html

If you click the [src] link on that page, you'll see that the implementation of std::mem::swap uses unsafe. I'm no Rust expert, but I'm also not aware of a way to swap pointers without unsafe.

Re: The Rust compiler is still getting faster

#93

Post by carlmr »

commit wrote: post #92
carlmr wrote: post #82

Earlier quoted context omitted.

>For example I discovered recently that you cannot swap two pointer variables in rust without using unsafe. Surely there is some deep reason for that, that is really important until they can figure out how to fix it in some future Rust version, which they undoubtedly will. Wouldn't this work? https://doc.rust-lang.org/std/mem/fn.swap.html >But then i see that there is actually some progress in Rust to relax some of t…

>Wouldn't this work? https://doc.rust-lang.org/std/mem/fn.swap.html If you click the [src] link on that page, you'll see that the implementation of std::mem::swap uses unsafe. I'm no Rust expert, but I'm also not aware of a way to swap pointers without unsafe.

Ah I thought you meant without using unsafe yourself.

The goal of unsafe is to reduce the attack surface where you need to check your code more thoroughly. So by encapsulating this in a library function that's well tested you don't need to create such a section yourself. But if you do need unsafe it's not wrong, and if only a small part of your code needs unsafe then it's much easier to verify that your code is doing what it should.

Re: The Rust compiler is still getting faster

#94

Post by pjmlp »

maximilianburke wrote: post #62
pjmlp wrote: post #46

Earlier quoted context omitted.

Only when incremental compiler, incremental linking and pre-compiled headers are disabled.

It's not just the compiler but also the system SDKs that play a significant role in the compile times. In an apples-to-apples comparison (ie: same pre-processed code content) from circa 2015, Visual Studio's compilation was about the same speed as clang and faster than GCC, without the use of incremental compilation / incremental linking / PCH. It's very easy to dig yourself into a hole when developing for Windows be…

Kind of true, but the common use of distributing binaries means that one can also take care to modularize the application across lib, DLL or COM modules, thus reducing even more what actually gets compiled from scratch.

Naturally one can do that across UNIXes as well, they just historically seem not to have invested that much either in incremental linking nor in a proper pre-compiled header model, hence no one really uses them. At least that is my perception.

In any case modules are in now, so lets see how they evolve.

Re: The Rust compiler is still getting faster

#95

Post by oefrha »

fluffything wrote: post #5
sddfd wrote: post #2

This is great! Many dev hours are spend waiting for the compiler, every second counts! The second order effects are even worse. After a minute, the programmer will start thinking about other things, running flow. If compiles regularly take 5min, devs will leave their desks (and honestly, who can blame them for it).

I just use `cargo watch -x test -d 5` to run all my tests after I pause modifying code for 5 seconds. My editor (emacs) uses `cargo watch -x check -d 0.5` to run `cargo check` (which is blazing fast for incremental edits) to type check all the code and show "red squiggles" with the error messages inline. So my interactive workflow with Rust is only edit-"type check"-edit-"type check" where "type check" takes often le…

You can easily get linking errors when using bindgen to link to C libraries (usually in the form of -sys crates). In fact, I once spent a fair bit of time trying to figure out how to link to FFmpeg (libav*) on Windows, without success.

Re: The Rust compiler is still getting faster

#96

Post by blub »

carlmr wrote: post #83
oblio wrote: post #77

Earlier quoted context omitted.

Don't all static analyzers do that? Or they're just ignored, which happened in almost every large project I've ever seen.

Exactly. C++ code has so much variability that 90% of static analyzer warnings are worthless. So people ignore them and create terrible unsafe code.

My comment had nothing to do with C++ or any other particular language, you're using it as a red herring to divert attention from the fact that the Rust analyzer requires changing the way you write code, making it not awesome, but ordinary.

And speaking of C++, it's obvious that Rust programmers take a very adversarial stance towards C++, but you should be instead thankful that it exists, because without it many of those marketing articles would lose their meaning.

"Slow compiler is now less slow, but probably still slower than those of other languages you know" doesn't quite have the same effect.

Re: The Rust compiler is still getting faster

#97

Post by carlmr »

blub wrote: post #96
carlmr wrote: post #83

Earlier quoted context omitted.

Exactly. C++ code has so much variability that 90% of static analyzer warnings are worthless. So people ignore them and create terrible unsafe code.

My comment had nothing to do with C++ or any other particular language, you're using it as a red herring to divert attention from the fact that the Rust analyzer requires changing the way you write code, making it not awesome, but ordinary. And speaking of C++, it's obvious that Rust programmers take a very adversarial stance towards C++, but you should be instead thankful that it exists, because without it many of t…

C++ is the obvious benchmark language because it's widespread and has the same benefits (manual memory management, native, high control).

High reliability, static checked any language requires you to write code a certain way. C++ is just the most prominent and meaningful example here.

ADA would be another good comparison, but I haven't worked with it on large projects so I can't say much about compile speed. It has a lot of similarities in being a safer C++. It felt similarly restrictive to Rust though.

I think the fact of the matter is that it you want safety you need some kind of restrictions and your compiler needs to do some extra work (ADA and Rust) or you need to look at the whole toolchain that o of necessary to achieve safety (in C++ this is very long if you run your tests with Static analyzers, ASan and Msan, too which remove some of the bugs Rust tells you about).

Re: The Rust compiler is still getting faster

#99

Post by winter_blue »

fluffything wrote: post #3

The rust compiler now has features that few or none C and C++ compilers have: incremental compilation within a single translation unit, pipelined compilation, multi-threaded and lazy query-based compilation, ... Implementing each of these features have required whole program refactorings in a large-scale codebase performed by few individuals while hundreds of other developers where simultaneously evolving the softwar…

Wow, this is really encouraging. I was debating whether to use Rust or Kotlin for developing a new programming language, and after reading your comment, I'm leaning a little bit closer to Rust. I had previously broken down the pros for Kotlin to 6 points:

1. You get access to all of the JVM libraries.

2. You don't have to work within the confines of the borrow checker.

3. Kotlin “Common” targets 3 platforms: LLVM, JVM, and JS (whereas, Rust only targets LLVM).

4. Kotlin is probably a more terse language, and suitable for doing algorithm / problem-solving interviews, and therefore a good one to be fluent in.

5. Kotlin's greater industry traction means that it might be more useful professionally. Outside of Android, I've also heard of servers/back-ends being written in Kotlin.

6. ANTLR is well-documented, compared to LALRPOP (the best existing Rust parser generator), and ANTLR targets/generates code for several mainstream languages (versus only Rust with LALRPOP). ANTLR is also probably a more useful skill to have for future jobs/projects.

But despite of all the pluses of using Kotlin, I'm still leaning a bit closer to Rust, because of all the good things I'm hearing about it.

Re: The Rust compiler is still getting faster

#100

Post by fluffything »

losvedir wrote: post #65
fluffything wrote: post #48

Earlier quoted context omitted.

> It’s #3 that I would suggest is a good candidate for stdlib inclusion You started arguing that the Rust compiler should expose its internal data-structures, which it does, and somehow ended arguing that the Rust standard library should expose spatial data-structures for geometry processing. I have no idea how you got from one to the other, but writing a huge wall of text full of disorganized thoughts shows very lit…

This is unnecessarily mean, and ironically you're the one showing little appreciation since you didn't even read derefr's initial post closely enough. They argued that the data structures (maps, graphs, etc) and algorithms used by the compiler could be included in the stdlib, not that the particular internal instantiations of them should be accessible. The whole point was the compiler uses a directed acyclic graph, s…

The argument is correct in that the concept of vectors, deques, hash-tables and graphs hasn't changed much and an API for them could be exposed.

The standard library exposes an API for some of them, and you can access the internal data-structures of the Rust by adding a line to your program to import them (requires unstable Rust) or by adding a line to your Cargo.toml to load them from crates.io.

The argument is flawed in that these are set in stone in any practical way. This is because Rust binaries do not run on a whiteboard, they run on real hardware, which means that there are thousands of different ways to efficiently implement these data-structures depending on what your precise use case is, and most of them affect their API design. For example, while Rust is lucky enough to provide a HashTable with a Google Swisstable-compatible API, C++ std::unordered_ containers are not, and cannot be upgraded.

The argument is also flawed into assuming that Rust is a static language. It isn't, the language is evolving, and as the language evolves the requirements on the internal data-structure changes, resulting in changes to the algorithms and data-structures being used (not only their implementation).

For example, at the end of last year Rust shipped non-lexical lifetimes, which uses a complete different borrow checking algorithm, data-structures, etc. The old ones just are not compatible with it.

Then there is also the fact that many people work on improving the performance on the Rust compiler. This means that the graph data-structures used are often not generic, but exploit the graph structure and the precise type of the graph nodes. As these data-structures are parallelized, made cache friendly, allocation-free, exploit usage of other platform-specific features like thread-locals, atomics, ... their APIs often changes. Also, often somebody just "discovers" that there are better algorithms and data-structures for implementing one pass, and they just change them.

Putting these in the standard library incurs a massive cost since this makes them almost impossible to evolve inside the Rust compiler, adds a huge maintenance cost, etc. And for what value? If you want to precisely use what the Rust compiler uses, you can already add a line to your program to import that from somewhere else that makes it clear that these are not stable. If you want general graph data-structures, chances are that your graph won't look like the ones used by the rust compiler. There are hundreds of crates for manipulating graphs in crates io, depending on how big the graphs are, whether you can exploit some structure, the common operations that you want to do with them, etc.

Sure, on the whiteboard, all of them probably have the same or similar complexity-guarantees, but on real hardware constant factors make a big difference for real problems.

Post reply on HN