Rust’s dependencies are starting to worry me • Hacker News Board

Re: Rust’s dependencies are starting to worry me

#471

Post by hu3 » Sat, May 10, 2025, 2:07 PM UTC

constantcrying wrote: post #413

rs186 wrote: post #399

Earlier quoted context omitted.

> You still can not articulate why these dependencies are unnecessary. No, because I don't have to answer that question. I can simply choose not to use this project, like what I do with npm projects. There is a project that's 500kb in code with 120 dependencies, when another one is 100kb with 10 dependencies that's also well maintained? I'll choose the latter without question, as long as it satisfies my needs. I don'…

Why are you complaining that a project you do not care about is using 13 dependencies, all of which, to your knowledge, are absolutely essential for the functionality? >There is a project that's 500kb in code with 120 dependencies And therefore some project using 13 dependencies is doing it wrong? What are you on about. Obviously there is an enormous abuse of dependencies in the JS ecosystem, who cares?

Their original complaint was about the project taking 20GB of disk space to compile.

Also they did point out that the parser depends on a serialisation library, so you're also mistaken about parent thinking the dependencies are necessary.

On another note, this pervasive kind of passive aggressive, hand-wavy, tribalistic, blind defense of certain technologies speak volumes about their audiences.

Re: Rust’s dependencies are starting to worry me

#472

Post by whstl » Sat, May 10, 2025, 2:11 PM UTC

stevenhuang wrote: post #430

whstl wrote: post #396

Earlier quoted context omitted.

Mainly because in some libs some code is activated at runtime. A lot of the bloat comes from functionality that can be activated via flags, methods that set a variable to true, environment variables, or even via configuration files.

When talking about LTO we don't expect it to be removing code used in runtime. Such code is not dead code, by definition. If you want to disable certain runtime features, you'd do so with feature flags.

Sure, but I'm talking about bloat in libraries that don't get LTO'd. If there are no feature flags and no plugin functionality, LTO can't do its job. There are plenty of non-core libraries like this.

Re: Rust’s dependencies are starting to worry me

#473

Post by [deleted] » Sat, May 10, 2025, 2:14 PM UTC

okanat wrote: post #409

whstl wrote: post #398

Earlier quoted context omitted.

So what happens if the user passes an url containing ftp:// or even https:// to stdin? Or is this an HTTP only library?

See, the trait system in Rust actually forced you to discover your requirements at a very core level. It is not a bug, but a feature. If you need HTTPS, then you need to include the code to do HTTPS of course. Then LTO shouldn't remove it. If your library cannot parse FTP, either you enable that feature, add that feature, or use a different library.

[deleted]

Re: Rust’s dependencies are starting to worry me

#474

Post by burntsushi » Sat, May 10, 2025, 2:24 PM UTC

neonsunset wrote: post #465

iTokio wrote: post #300

Earlier quoted context omitted.

I think this is partially true, but more nuanced than just saying that Rust std lib is lacking. Compared to go and c#, Rust std lib is mostly lacking: - a powerful http lib - serialization But Rust approach, no Runtime, no GC, no Reflection, is making it very hard to provide those libraries. Within these constraints, some high quality solutions emerged, Tokio, Serde. But they pioneered some novel approaches which wou…

To be fair I think Rust has very healthy selection of options for both, with Serde and Reqwest/Hyper being de-facto standard. Rust has other challenges it needs to overcome but this isn't one. I'd put Go behind both C#/F# and Rust in this area. It has spartan tooling in odd areas it's expected to be strong at like gRPC and the serialization story in Go is quite a bit more painful and bare bones compared to what you g…

> (because it does not allow writing sufficiently fast code in this area at this moment)

I don't think that's why. Or at least, I don't think it's straight-forward to draw that conclusion yet. I don't see any reason why the lazy DFA in RE2 or the Rust regex crate couldn't be ported to Go[1] and dramatically speed things up. Indeed, it has been done[2], but it was never pushed over the finish line. My guess is it would make Go's regexp engine a fair bit more competitive in some cases. And aside from that, there's tons of literal optimizations that could still be done that don't really have much to do with Go the language.

Could a Go-written regexp engine be faster or nearly as fast because of the language? Probably not. But I think the "implementation quality" is a far bigger determinant in explaining the current gap.

[1]: https://github.com/golang/go/issues/11646

[2]: https://github.com/matloob/regexp

Re: Rust’s dependencies are starting to worry me

#475

Post by hu3 » Sat, May 10, 2025, 2:29 PM UTC

arp242 wrote: post #287

3.6M lines of code seems so much that it sets off my "are you sure that's counted right?" alarm. I'm not very familiar with Rust, but all of Go is 1.6M lines of Go code. This includes the compiler, stdlib, tests for it all: the lot. Not that I doubt the sincerity of the author of course, but maybe some irrelevant things are counted? Or things are counted more than once? Or the download tool does the wrong thing? Or t…

This other person wrote their own async runtime and web server from scratch to reduce bloat, and their rust app still vendors 2 million lines of code:

https://news.ycombinator.com/item?id=43942055

Re: Rust’s dependencies are starting to worry me

#476

Post by timeinput » Sat, May 10, 2025, 2:32 PM UTC

hombre_fatal wrote: post #455

kibwen wrote: post #448

Earlier quoted context omitted.

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

Which glob crate? https://crates.io/search?q=glob I went to page 8 and there were still glob libraries.

That's much more a statement about the search function on crates.io than it is the number of glob crates. I think if you have the standard glob crate as a dependency you show up in that search.

Re: Rust’s dependencies are starting to worry me

#477

Post by timeinput » Sat, May 10, 2025, 2:34 PM UTC

hu3 wrote: post #453

kibwen wrote: post #448

Earlier quoted context omitted.

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

glob was just an example. They weren't asking about a specific crate. Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. A bad example all around.

I think the parent was suggesting comparing and contrasting the glob dependency in rust, and npm. The one off isn't useful, but picking ten random, but heavily used packages probably is. The parent didn't really mention what the node version looked like though.

The npm glob package has 6 dependencies (those dependencies have 3+ dependencies, those sub dependencies have 6+ dependencies, ...)

As you point out the rust crate is from the official repo, so while it's not part of the standard library, it is maintained by the language maintenance organization.

Maybe that could make it a bad example, but the npm one is maintained by the inventor of npm, and describes him self as "I wrote npm and a pretty considerable portion of other node related JavaScript that you might use.", so I would say that makes it a great example because the people who I would expect care the most about the language are the package maintainers of these packages, and are (hopefully) implementing what they think are the best practices for the languages, and the eco-systems.

Re: Rust’s dependencies are starting to worry me

#478

Post by GuB-42 » Sat, May 10, 2025, 2:37 PM UTC

The (terrible) solution that we are seeing now is generative AI. Instead of importing a library, you ask an AI to write the code for you, the AI most likely has ingested a library that implements the features you need and will essentially copy-paste that part into your code, transforming it so that it matches the rest of your code.

I believe that it causes more problems than it solves, but it can be a solution to the problem of adding thousands of lines of code of dependency when you could write a 10-line function yourself.

Of course, the proper thing to do is not to be the wrong kind of lazy and to understand what you are doing. I say the wrong kind of lazy because there is a right kind of lazy, and it is about not doing things you don't need to, as opposed to doing them poorly.

Re: Rust’s dependencies are starting to worry me

#479

Post by tmtvl » Sat, May 10, 2025, 2:39 PM UTC

f_devd wrote: post #400

whstl wrote: post #398

Earlier quoted context omitted.

So what happens if the user passes an url containing ftp:// or even https:// to stdin? Or is this an HTTP only library?

Depends on what is desired, in this case it would fail (through the `?`), and report it's not a valid HTTP Uri. This would be for a generic parsing library that allows for multiple schemes to be parsed each with their own parsing rules. If you want to mix schemes you would need to be able to handle all schemes; you can either go through all variations (through the same generics) you want to test or just just accept t…

If you want to mix schemes you should just mix schemes.

  let uri: Uri = parse_uri(get_uri_from_stdin()) or fail;

Re: Rust’s dependencies are starting to worry me

#480

Post by Measter » Sat, May 10, 2025, 2:41 PM UTC

zahlman wrote: post #285

elabajaba wrote: post #268

Earlier quoted context omitted.

Part of the rust dependency issue is that the compiler only multithreads at the crate level currently (slowly being improved on nightly, but there's still some bugs before they can roll out the parallel compiler), so most libraries split themselves up into a ton of small crates because otherwise they just take too long to compile. edit: Also, `cargo-vet` is useful for distributed auditing of crates. There's also `car…

> so most libraries split themselves up into a ton of small crates because otherwise they just take too long to compile. In practice, does this make it feasible to pick and choose the pieces you actually need?

It can do. Additionally, because each part is now smaller it's now easier to ensure that each part, in isolation, does what it says on the tin. It also means that other projects can reuse the parts. An example of the last point would be the Regex crate.

Regex is split into subcrates, one of which is regex-syntax: the parser. But that crate is also a dependency of over 150 other crates, including lalrpop, proptest, treesitter, and polars. So other projects have benefited from Regex being split up.