Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

211–220 of 593 posts

Re: Rust’s dependencies are starting to worry me

#211

Post by rapnie »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

OTOH it also depends on the architecture you build. If you have a local-first thick client the initial install of 800 MB is less relevant if after install you communicate on a tightly controlled (by you) p2p networking stack, but take on heavy dependencies in the UI layer to provide you e.g. infinite collaborative canvas based collaboration and diagramming.

Re: Rust’s dependencies are starting to worry me

#212

Post by vzaliva »

They should take a look at OPAM (OCaml’s package manager). There was a really impressive talk at the OCaml Workshop at POPL or ICFP a couple of years ago about how it works. Basically, they have a huge CI infrastructure and keep all versions of every package ever published. So, once you’ve found the right set of dependencies for your project, you can be sure the exact versions will always be available via OPAM.

Re: Rust’s dependencies are starting to worry me

#213

Post by echelon »

Daishiman wrote: post #182
echelon wrote: post #69

Earlier quoted context omitted.

bigstrat2003's argument is approximately "Python is batteries included" My counter argument is that the "batteries included" approach tends to atrophy and become dead weight. Your counter seems to be "that's not an argument, that's just Rust hype." Am I interpreting you correctly? Because I think my argument is salient and correct. I don't want to be stuck with dated APIs from 20 years of cruft in the standard librar…

Your critique doesn't match the reality of Python users. There is a single datetime library. It covers 98% of use cases. If you want the final 2% with all the bells and whistles you can download it if you wish. There is a single JSON library. It's fast enough for almost anything you want. If you want faster libraries with different usability tradeoffs you can use one but I have never felt compelled to do so. Same thi…

The Python datetime library is legacy software and has terrible ergonomics, terrible safety, and heinous pitfalls. It's one of my least favorite in the industry.

https://dev.arie.bovenberg.net/blog/python-datetime-pitfalls...

But now you're stuck with it forever.

Python is packed full with this shit. Because it wasn't carefully planned and respect wasn't given to decisions that would last forever.

Python has two testing frameworks baked in, neither of which is good.

Python has historically had shitty HTTP libraries and has had to roll out several versions to fix the old ones because it couldn't break or remove the old ones. Newbies to the language will find those built in and will write new software with the old baggage.

Batteries included is a software smell. It's bad. You can't change the batteries even after they expire.

Re: Rust’s dependencies are starting to worry me

#214

Post by socalgal2 »

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a library that does the glob but no, this is a common and shit pattern in npm. I'd say easily 25% or more of all "your dependencies are out of date" messages are related to the argument parcing for the commandline tool in these libraries. That's just one example.

Also there's arguably design. Should a 'glob' library actually read the file system and give you filenames or should it just tell you if a string matches a glob and leave the reset to you? I think it's better design to do the later, the simplest thing. This means less dependencies and more flexibility. I don't have to hack it or add option to use my own file system (like for testing). I can use it with a change monitoring system, etc...

And, I'm sure there are tons of devs that like the glob is a "Do everything for me" library instead of a "do one specific thing" library which makes it worse because you get more "internet points" the more your library doesn't require the person using it to be a good dev.

I can't imagine it's any different in rust land, except maybe for the executable thing. There's just too many devs and all of them, including myself, don't always make the best choices.

Re: Rust’s dependencies are starting to worry me

#215

Post by mleonhard »

> I can't rewrite the world, an async runtime and web server are just too difficult and take to long for me to justify writing for a project like this (although I should eventually just for a better understanding).

I did this and it only solved half of the bloat:

https://crates.io/crates/safina - Safe async runtime, 6k lines

https://crates.io/crates/servlin - Modular HTTP server library, threaded handlers and async performance, 8k lines.

I use safina+servlin and 1,000 lines of Rust to run https://www.applin.dev, on a cheap VM. It serves some static files, a simple form, receives Stripe webooks, and talks to Postgres and Postmark. It depends on some heavy crate trees: async-fs, async-net, chrono, diesel, rand (libc), serde_json, ureq, and url.

2,088,283 lines of Rust are downloaded by `cargo vendor` run in the project dir.

986,513 lines using https://github.com/coreos/cargo-vendor-filterer to try to download only Linux deps with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu`. This still downloads the `winapi` crate and other Windows crates, but they contain only 22k lines.

976,338 lines omitting development dependencies with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu --keep-dep-kinds=normal`.

754,368 lines excluding tests with `cargo vendor-filterer --platform=aarch64-apple-darwin --exclude-crate-path='*#tests' deps.filtered`.

750k lines is a lot to support a 1k-line project. I guess I could remove the heavy deps with another 200 hours of work, and might end up with some lean crates. I've been waiting for someone to write a good threaded Rust Postgres client.

Re: Rust’s dependencies are starting to worry me

#216

Post by andrepd »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

Dead code elimination means binary size bloat does not follow from dependency bloat. So this point is pretty much invalid for a compiled language like Rust.

Re: Rust’s dependencies are starting to worry me

#217

Post by themk »

QuadmasterXLII wrote: post #138

I wonder how much good a “dependency depth” label on packages would do, at the crates.io level. Like, a package can only depend on a package with a lower declared dependency depth than it, and packages compete to have a low dependency depth as a badge.

Everytime I daydream about my perfect language, this is one of the features I think about.

Re: Rust’s dependencies are starting to worry me

#218

Post by Alupis »

int_19h wrote: post #165
Alupis wrote: post #93

Earlier quoted context omitted.

So we reinvent Java's bloated SDK again, with all of the "javax" packages. What's old is new?

Well, it turned out that the alternative is even worse, so... let's chalk it down to learning experience.

I'm outside of the Rust community, so my two cents are worthless - but in this thread it seems a lot of people are actually wanting a defacto app framework, not necessarily a bloated "kitchen sink" style stdlib.

The stdlib probably should remain simple, in my opinion. The complexity should be optional.

Re: Rust’s dependencies are starting to worry me

#219

Post by wpollock »

All the comments and suggestions for improving rust dependency handling seem useful to me. To deal with dependency sprawl now, until the situation changes, I use a number of tools. To avoid having to set this up for each new project, I've made a template project that I simply unzip to create new rust projects.

The tools I have found useful are:

cargo outdated # check for newer versions of deps

cargo deny check # check dependency licenses

cargo about # generate list of used licenses

cargo audit # check dependencies for known security issues

cargo geiger # check deps for unsafe rust

I haven't found a cargo tool I like for generating SBOMs, so I installed syft and run that.

cargo install-update # keep these tools updated

cargo mutants # not related to deps, but worth a mention, used when testing.

Having configured all these tools once and simply unzipping a template works well for me.

Suggestions for different or additional tools welcome!

Disclaimer: I'm not a professional rust developer.

Re: Rust’s dependencies are starting to worry me

#220

Post by skydhash »

philsnow wrote: post #183

> do I even need this crate at all? 35 lines later I had the parts of dotenv I needed. I'm not saying you copy-pasted those 35 lines from dotenvy, but for the sake of argument let's say you did: now you can't automatically benefit from dotenvy patching some security issue in those lines.

If you REALLY need such update, you can easily subscribe to updates from the mainstream project (in whatever way it allows) and patch your version when that rare situation occurs.
Post reply on HN