Live data from Hacker News

Thoughts on Rust bloat

raphlinus.github.io

161–170 of 313 posts

Re: Thoughts on Rust bloat

#161
post #115

Earlier quoted context omitted.

Who is "the community" and who isn't? Are you sure you trust the core language developers more than people writing libraries?

More often than not, absolutely. It requires a much higher level of competence to design a language and develop a functional compiler than it does to design libraries. There are packages in Node with millions of downloads, packages that are basically ubiquitous in certain domains that are riddled with bugs, with terrible interfaces and documentation. I can even think of libraries I've worked with in Java/C/.net/etc t…

Language design and compiler implementation are definetely high-competency skills, but they don't necessarily correlate with library design skills.

For instance, http client and server libraries are often in this gray area of uncertainty about whether they should be in the stdlib or not. Is this something language designers or compiler implementors have a lot of experience in? I would say not; sending and serving http requests are not something compilers need to do. Or take GUI libraries, what do language designers know about that?

I also know of many bad third party libraries, but there are tons of examples of really awful parts of standard libraries. The original date/time APIs in Java are a mess (they finally fixed this by in essence bringing in a third party API). The ssl bindings in the Ruby stdlib were a common source of bugs back when I was paying attention to this (maybe they've fixed it), same thing for the built in http stuff. Someone else mentioned the similar weakness of python's built in http, such that most people use a third party library instead. Even the java collections APIs are pretty poor such that people often augment them with things like guava or apache libraries.

My point is just that developing good libraries is a hard thing and I don't see any reason to think language designers or compiler maintainers are any better (or worse!) at it than other people. There isn't really a shortcut, you can't just cede authority to the powers that be on the core language teams, you just have to evaluate the quality of libraries for your use case yourself.

Re: Thoughts on Rust bloat

#162

Earlier quoted context omitted.

Yeah, see, I think it's anachronistic and odd that Java comes with GUI stuff included! That seems niche and better relegated to a library to me.

It wasn't, though, in 1995. Though it is perhaps less fashionable now, it was certainly something a lot of programs used at the time.

This is why I chose the word "anachronistic". It seems of another time, because it is. It's definitely hard to figure out what will and won't be timeless, but it isn't hard to look back with hindsight and point out things that definetely weren't.

Re: Thoughts on Rust bloat

#163
post #44
post #39

Earlier quoted context omitted.

Cargo compiles dependencies in parallel so if you have lots of cores you'll hear your fans spin up. I see a lot of difference between my i5 laptop and i9 desktop.

As a case in point, I have a project that uses amethyst and nalgebra (and their 200 transitive dependencies). On upgrading from an OC i5-4670k to a r9-3900x, my compile times for a clean release build went from 20 minutes to 2.

But two minutes is still a lot of time to wait for a build, especially if you’re doing gamedev and want to prototype something fast.

It seems nalgebra is the culprit here: because Rust doesn’t yet support const generics, it has to use some hacky type-level metaprogramming to represent numbers, and that will definitely destroy build times.

Re: Thoughts on Rust bloat

#164

Earlier quoted context omitted.

I think not anymore since java.time - which is incredibly similar to Joda - came around? This is actually an illustrative example of the process I like and hope rust will develop over time: let the community reach a consensus on the best third party libraries things, then consider pulling them in to or at least taking the best parts of their APIs for the standard library.

But won’t the community converge on that anyway, and in a similar amount of time? So why add it to the stdlib?

Stability? Guarantee that the api will continue to work and that no one will wake up one morning and left-pad the entire ecosystem?

Re: Thoughts on Rust bloat

#165

Earlier quoted context omitted.

I think not anymore since java.time - which is incredibly similar to Joda - came around? This is actually an illustrative example of the process I like and hope rust will develop over time: let the community reach a consensus on the best third party libraries things, then consider pulling them in to or at least taking the best parts of their APIs for the standard library.

But won’t the community converge on that anyway, and in a similar amount of time? So why add it to the stdlib?

There's a cost to discovering the consensus choices that I think inclusion in a standard lib minimizes. But there may be other similarly good ways to accomplish this. If I'm remembering correctly, doesn't Rust have a set of libraries that aren't in the standard library but are somehow vouched for? Maybe that's a similarly good approach to solve the discovery problem, I'm not sure.

Re: Thoughts on Rust bloat

#166
post #42
post #12

It looks like the author is most bothered by compile times of dependencies. Cargo needs to do better with shared caches (so you compile each dep at most once per machine) or ability to get precompiled crates (so you don't even compile it). Incremental improvements of compiler speed or trimming of individual dependencies won't bring the 10x improvement it needs.

Have you tried sccache [0]? It doesn’t always choose to cache a dependency, but it helps about 70% of the time. Anecdotally, it hastened a release build of a pretty standard CLI tool (with incremental compilation) by almost 4x. In the context of resource-constrained machines, one can always host it remotely on S3. (or mount an NFS share as the CARGO_TARGET_DIR, if you’re feeling adventurous or want fast CI) [0] https…

Can I use this across all my projects and machines and have shared, cached compilation of Rust crates?

Re: Thoughts on Rust bloat

#167

> There’s also an effort to analyze binary sizes more systematically. I applaud such efforts and would love it if they were even more visible. Ideally, crates.io would include some kind of bloat report along with its other metadata, [...] This is what I always wanted (for Rust as well as for C) but never got around to hack together myself. I dreamt it up more as a feature of cargo though, something like 'cargo stats'…

You might be looking for `cargo install cargo-bloat`

    $ cargo bloat
    Compiling ...
    Analyzing target/debug/mdbook

    File  .text     Size                 Crate Name
    0.5%   1.3% 166.4KiB                 regex ::captures_read_at
    0.5%   1.3% 165.2KiB                  idna unicode_normalization::tables::compatibility_fully_decomposed
    0.3%   0.7%  95.0KiB               ammonia html5ever::tree_builder::TreeBuilder::step
    0.3%   0.7%  92.5KiB                  idna unicode_normalization::tables::canonical_fully_decomposed
    0.2%   0.5%  64.6KiB               unicase unicase::unicode::map::lookup
    0.2%   0.5%  63.4KiB                  idna unicode_normalization::tables::is_combining_mark
    0.2%   0.4%  55.8KiB                 regex  as core::iter::traits::iterator::Iterator>::next
    0.2%   0.4%  55.5KiB                 regex regex::re_unicode::Regex::find_at
    0.1%   0.3%  41.7KiB unicode_normalization unicode_normalization::tables::composition_table
    0.1%   0.3%  36.2KiB               rand_hc rand_hc::hc128::Hc128Core::sixteen_steps
    0.1%   0.3%  33.8KiB                 regex regex::re_unicode::Regex::shortest_match_at
    0.1%   0.3%  32.4KiB               rand_hc ::generate
    0.1%   0.2%  31.4KiB               ammonia html5ever::tokenizer::Tokenizer::step
    0.1%   0.2%  24.5KiB                  idna unicode_normalization::tables::canonical_combining_class
    0.1%   0.2%  21.8KiB                 regex aho_corasick::ahocorasick::AhoCorasick::find
    0.1%   0.2%  21.6KiB                 regex aho_corasick::ahocorasick::AhoCorasick::find
    0.1%   0.2%  21.0KiB                  clap clap::app::parser::Parser::get_matches_with
    0.1%   0.2%  20.3KiB                    ws ws::io::Handler::handle_queue
    0.1%   0.1%  19.0KiB                    ws ws::connection::Connection::read_frames
    0.1%   0.1%  18.7KiB            env_logger termcolor::Ansi::write_color
    35.3%  91.6%  11.4MiB                       And 58189 smaller methods. Use -n N to show more.
    38.6% 100.0%  12.5MiB                       .text section size, the file size is 32.4MiB

Re: Thoughts on Rust bloat

#168
post #44

Earlier quoted context omitted.

As a case in point, I have a project that uses amethyst and nalgebra (and their 200 transitive dependencies). On upgrading from an OC i5-4670k to a r9-3900x, my compile times for a clean release build went from 20 minutes to 2.

But two minutes is still a lot of time to wait for a build, especially if you’re doing gamedev and want to prototype something fast. It seems nalgebra is the culprit here: because Rust doesn’t yet support const generics, it has to use some hacky type-level metaprogramming to represent numbers, and that will definitely destroy build times.

2 minutes for a full build from scratch. Incremental builds afterwards take seconds, though unfortunately linking of big projects can still sometimes takes up to around a minute.

Re: Thoughts on Rust bloat

#169
post #138

Earlier quoted context omitted.

Opinions on this are a dime a dozen. You often see the reverse of it too, for example, you might have heard that "Python's standard library is where things go to die." You could just as easily call that a "terrible error." The fact that Python's standard library has an HTTP client in it, for example, doesn't stop everyone from using requests (and, consequently, urllib3) for all their HTTP client needs. So despite the…

I feel like a reasonable middle ground to these issues is for communities to perhaps embrace “metapackages” that serve as community maintained “standard libraries”. In the R community we have tidyverse, which is practically a full mirror of R’s standard libs at this point. These metapackage communities can then focus on interoperability of constituent packages w/o overburdening the standard libraries, and core langua…

Several attempts have been made at this in Rust in the past, but few people use them. It just adds even more dependencies that you’re not actually using.

Re: Thoughts on Rust bloat

#170

Earlier quoted context omitted.

But won’t the community converge on that anyway, and in a similar amount of time? So why add it to the stdlib?

There's a cost to discovering the consensus choices that I think inclusion in a standard lib minimizes. But there may be other similarly good ways to accomplish this. If I'm remembering correctly, doesn't Rust have a set of libraries that aren't in the standard library but are somehow vouched for? Maybe that's a similarly good approach to solve the discovery problem, I'm not sure.

There are libraries that are maintained by the project itself, but are not part of the standard library, yes.
Post reply on HN