I think that https://blessed.rs does a pretty good job of providing recommendations for things that probably can't be crammed into the standard library, but which you'll almost certainly end up needing at one point or another. I honestly like that system a lot, it makes it so that the only packages you need to worry much about are usually doing something rather specific.
Rust’s dependencies are starting to worry me
81–90 of 593 posts
Re: Rust’s dependencies are starting to worry me
#82Earlier quoted context omitted.
There should be a second stdlib with relaxed stability guarantees. Don't fill the normal stdlib full of cruft that can never be changed again.
Actually, a proposal for exactly this was published yesterday: https://github.com/rust-lang/rfcs/pull/3810 It's unfortunate that the response so far hasn't been very positive
That proposal is not exactly this; that seems to propose a "blessed crates" namespace which includes popular open-source libraries. I read this proposal as a Python-style batteries-included stdlib.
Re: Rust’s dependencies are starting to worry me
#83Similar feeling here. Cargo makes it so simple to add tons of dependencies that it is really hard not to do it. But that does not stop here: even if I try to be careful with adding dependencies, a couple dependencies are likely to pull tens of transitive dependencies each. "Then don't depend on them", you say. Sure, but that means I won't write my project, because I won't write those things from scratch. I could prob…
> Sure, but that means I won't write my project, because I won't write those things from scratch. You need to think a bit harder about that, to help you decide whether your position is rational.
This confuses me as well. Is the implied solution to choose a language where you are forced to write those things from scratch?
Re: Rust’s dependencies are starting to worry me
#84Earlier quoted context omitted.
ok, I'm hooked - how is setting an env var in the current process unsafe? My gut says it's not unsafe in a memory-ownership sense, but rather in a race condition sense? whatever the issue is, "setting an env var is unsafe" is so interesting to me that I'm now craving a blog post explaining this
It's a long standing bug, setenv and unsetenv are not thread-safe https://www.evanjones.ca/setenv-is-not-thread-safe.html
I honestly think using setenv is just a terrible idea.
Re: Rust’s dependencies are starting to worry me
#85IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos.
This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painstakingly included parts of into your size constrained environment (fit it on a floppy). You probably picked apart that library and pulled the bits you needed, integrating them into your builds to be as small as possible.
Today we pile libraries on top of libraries on top of libraries. Its super easy to say `import foolib`, then call `foolib.do_thing()` and just start running. Who knows or cares what all 'foolib' contains.
At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number.
In some cases the languages make this worse. Go and Rust, for example, encourage everything for a single package/mod to go in the same file. Adding optional functionality can get ugly when it would require creating new modules, but if you only want to use a tiny part of the module, what do you do?
The only real solution I can think of to deal with this long term is ultra-fine-grained symbols and dependencies. Every function, type, and other top-level language construct needs to declare the set of things it needs to run (other functions, symbols, types, etc). When you depend on that one symbol it can construct, on demand, the exact graph of symbols it needs and dump the rest for any given library. You end up with the minimal set of code for the functionality you need.
Its a terrible idea and I'd hate it, but how else do you address the current setup of effectively building the whole universe of code branching from your dependencies and then dragging it around like a boat anchor of dead code.
Re: Rust’s dependencies are starting to worry me
#86We need a term like “Mature” or similar for dependencies that are done. Mature dependencies have two characteristics: 1. Well defined scope 2. Infrequent changes Nomad has many of these (msgpack, envparse, cli, etc). These dependencies go years without changing so the dependency management burden rapidly approaches zero. This is an especially useful property for “leaf” dependencies with no dependencies of their own.…
I have a lot of sympathy for this viewpoint, but I also ask that we try to remind ourselves. We are asking for professionalism from hobby projects. If you want a mature protobuf implementation you should probably buy one. Expecting some guy/gal on the internet to maintain one for your for free seems ill advised.
Isn't that an argument _for_ having a "mature" label? To avoid the hobbyists who have no intention to maintain their thing?
Also there are lots of lovely projects maintained at high levels by hobbyists, and plenty of abandonware that was at some point paid for
Re: Rust’s dependencies are starting to worry me
#87Earlier quoted context omitted.
Actually, a proposal for exactly this was published yesterday: https://github.com/rust-lang/rfcs/pull/3810 It's unfortunate that the response so far hasn't been very positive
That proposal is not exactly this; that seems to propose a "blessed crates" namespace which includes popular open-source libraries. I read this proposal as a Python-style batteries-included stdlib.
What the OP proposes is not exactly a bigger stdlib, because they mention it should have "relaxed stability guarantees". Or is python allowed to change their stdlib in backwards-incompatible ways?
Re: Rust’s dependencies are starting to worry me
#88Earlier quoted context omitted.
I take bit less unstable dependencies over the total mess of C++ dependencies with CMake, shared libraries, version conflicts etc any time. There's probably also a bit of an illusion about C++ transitive dependencies due to them usually being precompiled (because compiling them is such pain).
The whole pkgconfig, cmake, autotools etc ecosystem is insane compared to how Rust and Go do things. It's part of the reason why software distribution on Linux has been pushed to using containers, removing the point of having shared libraries. I think Google with it's C++ replacement (Carbon) plans on doing it's own system.
From my point of view, the issue stems from developers wanting to control distribution. Fine if it's for your own usage, not really if you're planning for others to use it. You will find the most convoluted build system just because they have a pet platform they want to specially support making it hell to do anything on others.
It could be better, but the current solutions (npm, go, python,...) favor only the developers, not the maintainers and packagers.
Re: Rust’s dependencies are starting to worry me
#89Earlier quoted context omitted.
Best way is to have CI/CD systems only connected to the official internal repos. Devs can add whatever they feel like on their workstations but it will be a sad build server if they get pushed without permission.
s/Best way/The only safe way/ Anything else will get abused in the name of expediency and just-this-one-time. Also, the process for adding a crate/gem/module/library needs to be the same as anything else: license review, code review, subscription to the appropriate mailing list or other announce channel, and assignment of responsibility. All of these except code review can be really, really fast once you have the pro…
I agree that some amount of friction when including third party dependencies is a vital thing to push people to consider the value versus cost of dependencies (and license review, code review, channel subscriptions are all incredibily important and almost always overlooked), however how should this work for transitive dependendencies? And the dependencies of _those_ dependencies?
The dependency trees for most interpreted or source-distributed languages are ridiculous, and review of even a few of those seems practically impossible in a lot of development environments.
Re: Rust’s dependencies are starting to worry me
#90IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…
As far as I'm aware, LTO completely solves this from a binary size perspective. It will optimise out anything unused. You can still get hit from a build time perspective though.