Seems like a lot of this comes from rust defaulting to allowing several different versions of a library to be linked in ... there’s def some other pieces. But that seems like a biggie
Do you know how to not do this "default"? I would love to try it.
Thoughts on Rust bloat
41–50 of 313 posts
Re: Thoughts on Rust bloat
#42It looks like the author is most bothered by compile times of dependencies. Cargo needs to do better with shared caches (so you compile each dep at most once per machine) or ability to get precompiled crates (so you don't even compile it). Incremental improvements of compiler speed or trimming of individual dependencies won't bring the 10x improvement it needs.
Have you tried sccache [0]? It doesn’t always choose to cache a dependency, but it helps about 70% of the time. Anecdotally, it hastened a release build of a pretty standard CLI tool (with incremental compilation) by almost 4x.
In the context of resource-constrained machines, one can always host it remotely on S3. (or mount an NFS share as the CARGO_TARGET_DIR, if you’re feeling adventurous or want fast CI)
Re: Thoughts on Rust bloat
#43>the release binary is now 5.9M A typical smartphone ships with around 10,000 times this much storage capacity and enough RAM to hold it 100 times over. This is bloat? I mean, I get it, the binary used to be only 2MB, 1/3rd the size. But are numbers this low really worth worrying about? I think a GUI app in 6MB is hugely impressive. I genuinely thought he was going to say it was 100MB or something higher.
I think a GUI app in 6MB is hugely impressive. You think that because nobody cares about bloat anymore, so everything else is large.
If you want to code a large app in C with X Athena Widgets or Win32 you're welcome to do so.
Re: Thoughts on Rust bloat
#44It looks like the author is most bothered by compile times of dependencies. Cargo needs to do better with shared caches (so you compile each dep at most once per machine) or ability to get precompiled crates (so you don't even compile it). Incremental improvements of compiler speed or trimming of individual dependencies won't bring the 10x improvement it needs.
Cargo compiles dependencies in parallel so if you have lots of cores you'll hear your fans spin up. I see a lot of difference between my i5 laptop and i9 desktop.
As a case in point, I have a project that uses amethyst and nalgebra (and their 200 transitive dependencies).
On upgrading from an OC i5-4670k to a r9-3900x, my compile times for a clean release build went from 20 minutes to 2.
Re: Thoughts on Rust bloat
#45Earlier quoted context omitted.
Whether culture or language, if it is not addressed then expect similar results :(
The primary way I can see to address this is to make packaging more difficult. Are there other steps that a language can take to avoid this problem?
A few ideas:
1) Remove the ability to unpublish / yank crates. A published crate should be immutable, but the crate's metadata should always be updateable by the maintainer.
2) Improve the metadata that describes a crate so that it is easy to tell if a crate should be used. For example, is the crate beta quality? Was a serious error found in the crate and it needs to be marked as "not safe"? Is it a Long-Term Support release? Etc.
3) As a culture, disallow trivial crates. No "is-odd" or similarly low effort crates. These just add bloat since the have so little functionality compared to their overhead. If your crate's toml is larger than the crate's code, you are doing it wrong.
Re: Thoughts on Rust bloat
#46At the risk of being slightly tangential, I've been sorely wanting to air this particular grievance with Rust for some time. It's somewhat related, since the author mentions their package system. Its package ecosystem isn't nearly in the horrible state that node's is, but having a package system shouldn't be a substitute for designing a useful standard library for a language. I think that the attraction to 'small lan…
Isn't it at least partly explained by wanting to allow implementations of this functionality to evolve more rapidly than they could if they were in the stdlib and thus reach good solutions more rapidly? getopt in particular seems like something that programmers haven't reached a consensus on despite 40 years of experimentation.
Re: Thoughts on Rust bloat
#47Earlier quoted context omitted.
Performance culture has you measure the actual performance implications, then make an informed decision. Is the code on a performance-critical path? Maybe some of your serialization code is, but it's extremely unlikely that a dynamic dispatch when parsing command line args is the reason your app is slow. Also be aware that highly inlined code does nicely in microbenchmarks but might have significantly negative perfor…
> Also be aware that highly inlined code does nicely in microbenchmarks but might have significantly negative performance implications in a larger system when it blows out the I-cache. I see this assertion a lot, but I have never actually seen a system in which inlining that would otherwise be a win in terms of performance becomes a loss in a large system. LLVM developers seem to agree, because LLVM is quite aggressi…
Fiora refactored the MMU code emitted by Dolphin to a far jump, which had significant performance improvements over inlining the code [0]. She had an article about it in PoC || GTFO [1].
[0] https://dolphin-emu.org/blog/2014/09/30/dolphin-progress-rep...
[1] https://github.com/angea/pocorgtfo/blob/master/contents/issu...
Re: Thoughts on Rust bloat
#48Maybe rust core/std/alloc should built with many smaller crates that are cached better and have fewer dependencies?
Re: Thoughts on Rust bloat
#49Earlier quoted context omitted.
Whether culture or language, if it is not addressed then expect similar results :(
The primary way I can see to address this is to make packaging more difficult. Are there other steps that a language can take to avoid this problem?
AFAIK the real problem in node is not number of packages in itself, but number of independent trust relationships implied by transitive dependencies. Basically how many separate people's integrity and security practices are you counting on when you install your reqs?
So a language could:
- identify a blessed set of packages that don't imply separate trust relationships (a stdlib, or packages maintained or audited by the language team)
- require strong security practices and/or trust metrics from package authors
- clearly expose the full list of authors implicitly trusted by a requirements file
- offer community auditing tools, such as reproducible builds if delivering compiled files, web of trust tools, or diff tools
- run internal tests for suspicious packages (and don't talk about them)
If you frame the problem differently the solutions might be different, but I suspect there would still be a bunch of options.
Re: Thoughts on Rust bloat
#50Rust bloat is a serious issue that is not being taken seriously I think. I raised the issue about platform size in June: https://github.com/rust-lang/rust/issues/61978 and its actually gotten worse since then, significantly worse. In the 2 months since then the installer has increased from 203 MB to 299 MB. Also unbelieveably, Rust has failed to address package balkanization which I would say has ruined the Node comm…
Great job commenting on a four year old thread on the package manager instead of going to the package maintainers themselves.