Rust’s dependencies are starting to worry me • Hacker News Board

Re: Rust’s dependencies are starting to worry me

#281

Post by flomo » Sat, May 10, 2025, 4:21 AM UTC

mysterymath wrote: post #246

zozbot234 wrote: post #141

Earlier quoted context omitted.

> In the 80s the idea of a library of functionality was something you paid for, and painstakingly included parts of into your size constrained environment (fit it on a floppy). You probably picked apart that library and pulled the bits you needed, integrating them into your builds to be as small as possible. If anything, the 1980s is when the idea of fully reusable, separately-developed software components first beca…

You're talking about different 80s. On workstations and Unix mainframes, beasts like Smalltalk and Objective C roamed the Earth. On home computers, a resident relocatable driver that wasn't part of ROM was an unusual novelty.

Yeah, 1990s is more accurate. There was a huge market for COM controls and widget libs and a lot of that Obj-C stuff came with a price tag.

Re: Rust’s dependencies are starting to worry me

#282

Post by zahlman » Sat, May 10, 2025, 4:23 AM UTC

Orangeair wrote: post #81

I think that https://blessed.rs does a pretty good job of providing recommendations for things that probably can't be crammed into the standard library, but which you'll almost certainly end up needing at one point or another. I honestly like that system a lot, it makes it so that the only packages you need to worry much about are usually doing something rather specific.

Would love to see something like this for Python.

Re: Rust’s dependencies are starting to worry me

#283

Post by zahlman » Sat, May 10, 2025, 4:25 AM UTC

X0Refraction wrote: post #30

palata wrote: post #3

Similar feeling here. Cargo makes it so simple to add tons of dependencies that it is really hard not to do it. But that does not stop here: even if I try to be careful with adding dependencies, a couple dependencies are likely to pull tens of transitive dependencies each. "Then don't depend on them", you say. Sure, but that means I won't write my project, because I won't write those things from scratch. I could prob…

This was linked from the top comment on the Rust subreddit: https://wiki.alopex.li/LetsBeRealAboutDependencies I think it makes a good point that some of the difference here is just perception due to dependencies in C/C++ being less immediately visible since they're dynamically loaded. To some degree that is a plus though as you likely trust the maintainers of your OS distribution to provide stable, supported librari…

> This was linked from the top comment on the Rust subreddit: https://wiki.alopex.li/LetsBeRealAboutDependencies

It was also posted here, shortly before this thread: https://news.ycombinator.com/item?id=43934343

(And several times in the past, too.)

> I think it makes a good point that some of the difference here is just perception due to dependencies in C/C++ being less immediately visible since they're dynamically loaded.

The point wasn't so much about the loading mechanism, but about the fact that the system (especially on Linux) provides them for you; a good amount come pre-installed, and the rest go through a system package manager so you don't have to worry about the language failing to have a good package system.

Re: Rust’s dependencies are starting to worry me

#284

Post by meling » Sat, May 10, 2025, 4:27 AM UTC

lantastic wrote: post #160

meling wrote: post #80

Earlier quoted context omitted.

Yeah, while I’ve seen some great libraries that follow the practice of minimizing their dependencies, I’m a bit annoyed with the amount of dependencies that docker will bring along [1]. I’ve been on the lookout for alternatives for my docker needs, but the state of podman, buildah and some others that I checked is similar. They all bring in roughly the same number of dependencies… if anyone knows of a stripped down G…

Wow, that's massive. I guess it's inevitable that a popular piece of open-source software for end-users will be compelled to accrue dependencies due to popular demand for features that require them. I feel Telegraf made a good compromise: out of the box, it comes with a _ton_ of stuff[1] to monitor everything, but they make it possible to build only with pieces that you need via build tags, and even provide a tool to…

Thanks! That’s an interesting approach. Haven’t seen that before. I think a better approach (in a monorepo) might be to use separate go.mod files for each module, allowing the user to configure only the needed parts separately. But I haven’t seen it used much.

Re: Rust’s dependencies are starting to worry me

#285

Post by zahlman » Sat, May 10, 2025, 4:28 AM UTC

elabajaba wrote: post #268

palata wrote: post #169

Earlier quoted context omitted.

> some of the difference here is just perception due to dependencies in C/C++ being less immediately visible since they're dynamically loaded. Not in my case. I manually compile all the dependencies (either because I need to cross-compile, or because I may need to patch them, etc). So I clearly see all the transitive dependencies I need in C++. And I need a lot less than in Rust, by a long shot.

Part of the rust dependency issue is that the compiler only multithreads at the crate level currently (slowly being improved on nightly, but there's still some bugs before they can roll out the parallel compiler), so most libraries split themselves up into a ton of small crates because otherwise they just take too long to compile. edit: Also, `cargo-vet` is useful for distributed auditing of crates. There's also `car…

> so most libraries split themselves up into a ton of small crates because otherwise they just take too long to compile.

In practice, does this make it feasible to pick and choose the pieces you actually need?

Re: Rust’s dependencies are starting to worry me

#286

Post by arp242 » Sat, May 10, 2025, 4:30 AM UTC

philsnow wrote: post #183

> do I even need this crate at all? 35 lines later I had the parts of dotenv I needed. I'm not saying you copy-pasted those 35 lines from dotenvy, but for the sake of argument let's say you did: now you can't automatically benefit from dotenvy patching some security issue in those lines.

What security issue? It's just read file by line, split by =, and return or call setenv. This is not OpenSSL we're talking about.

Re: Rust’s dependencies are starting to worry me

#287

Post by arp242 » Sat, May 10, 2025, 4:32 AM UTC

3.6M lines of code seems so much that it sets off my "are you sure that's counted right?" alarm.

I'm not very familiar with Rust, but all of Go is 1.6M lines of Go code. This includes the compiler, stdlib, tests for it all: the lot.

Not that I doubt the sincerity of the author of course, but maybe some irrelevant things are counted? Or things are counted more than once? Or the download tool does the wrong thing? Or there's tons of generated code (syscalls?)? Or ... something? I just find it hard to believe that some dependencies for web stuff in Rust is twice all of Go.

Re: Rust’s dependencies are starting to worry me

#288

Post by zahlman » Sat, May 10, 2025, 4:37 AM UTC

wofo wrote: post #87

tadfisher wrote: post #82

Earlier quoted context omitted.

That proposal is not exactly this; that seems to propose a "blessed crates" namespace which includes popular open-source libraries. I read this proposal as a Python-style batteries-included stdlib.

What the OP proposes is not exactly a bigger stdlib, because they mention it should have "relaxed stability guarantees". Or is python allowed to change their stdlib in backwards-incompatible ways?

It does happen - after a deprecation period, usually small changes, but frequently (i.e. in every minor version there will surely be someone directly affected). More recently there were entire swaths of modules removed - still a conservative change, because we're talking mainly about support for obscure file formats and protocols that hardly anyone has used this century (see https://peps.python.org/pep-0594/ for details - I may be exaggerating, but not by a lot).

Historically this process has been mostly informal; going forward they're trying to make sure that things get removed at a specific point after their deprecation. Python has also now adopted an annual release cadence; the combination of that with the deprecation policy effectively makes their versioning into a pseudo-calver.

Re: Rust’s dependencies are starting to worry me

#289

Post by zahlman » Sat, May 10, 2025, 4:41 AM UTC

echelon wrote: post #52

bigstrat2003 wrote: post #37

Earlier quoted context omitted.

> We do not need to saddle Rust with garbage that will feel dated like Python's standard library. Python's standard library is a strength, not a weakness. Rust should be so lucky. It's wonderful to have basic functionality which is guaranteed to be there no matter what. Many people work in environments where they can't just YOLO download packages from the Internet, so they have to make do with whatever is in the stdl…

> Python's standard library is a strength, not a weakness. Rust should be so lucky. Rust is luckier. It has the correct approach. You can find every battery you need in crates.io. Python has had monstrosities like urllib, urllib2, http, etc. All pretty much ignored in favor of the external requests library and its kin. The standard library also has inconsistencies in calling conventions and naming conventions and it…

> The standard library also has inconsistencies in calling conventions and naming conventions and it has to support those *FOREVER*.

Not to mention abysmal designs inspired by cargo-cult "OOP" Java frameworks from the 90s and 00s. (Come on, folks. Object-oriented programming is supposed to be about objects, not about classes. If it were about classes, it would be called class-oriented programming.)

Re: Rust’s dependencies are starting to worry me

#290

Post by Avi-D-coder » Sat, May 10, 2025, 4:42 AM UTC

The solution is strong compile time and runtime guarantees about code behavior.

The author is right there's no way an individual can audit all that code. Currently all that code can run arbitrary build code at compile time on the devs machine, it can also run arbitrary unsafe code at runtime, make system calls, etc..

Software is not getting simpler, the abundance of high quality libraries is great for Rust, but there are bound to be supply chain attacks.

AI and cooperative auditing can help, but ultimately the compiler must provide more guarantees. A future addition of Rust should come with an inescapable effect system. Work on effects in Rust has already started, I am not sure if security is a goal, but it needs to be.