Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

181–190 of 593 posts

Re: Rust’s dependencies are starting to worry me

#181

Post by rs186 »

constantcrying wrote: post #164
rs186 wrote: post #152

Earlier quoted context omitted.

13 > 12 so over a dozen dependencies. If you look at acorn or babel/parser, they barely have any dependency. Repository size is directly related to how long it takes to run a build, which is extremely important if I were to contribute to the project. > Serde seems exactly a case where you absolutely should use an external dependency. I can't see any reason a parser has a hard dependency on a serialization library.

>13 > 12 so over a dozen dependencies. If you look at acorn or babel/parser, they barely have any dependency. Which ones are superfluous? There are good reasons to use dependencies. If someone has solved a problem you need to solve as well it is pointless to duplicate the effort. >Repository size is directly related to how long it takes to run a build, which is extremely important if I were to contribute to the proje…

I don't think there is any point in debating this, because apparently you are in the camp of "dependencies are ok", with or without a good reason, when a different camp is "avoid dependencies unless you really have to". You just provided an example of why dependencies explode like this.

> And because you can't see a reason there is none?

Somehow every other JS based parser doesn't do fancy serialization, as far as I can tell. You can come up with reasons of why one might need it, but as a user of the parser, I want the footprint to be small, and that's a requirement. In fact, that's one of the reasons I never used swc parser in my serious projects.

Re: Rust’s dependencies are starting to worry me

#182

Post by Daishiman »

echelon wrote: post #69
wolvesechoes wrote: post #66

Earlier quoted context omitted.

"Rust is doing it right." Standard response every time there is some criticism of Rust.

bigstrat2003's argument is approximately "Python is batteries included" My counter argument is that the "batteries included" approach tends to atrophy and become dead weight. Your counter seems to be "that's not an argument, that's just Rust hype." Am I interpreting you correctly? Because I think my argument is salient and correct. I don't want to be stuck with dated APIs from 20 years of cruft in the standard librar…

Your critique doesn't match the reality of Python users.

There is a single datetime library. It covers 98% of use cases. If you want the final 2% with all the bells and whistles you can download it if you wish. There is a single JSON library. It's fast enough for almost anything you want. If you want faster libraries with different usability tradeoffs you can use one but I have never felt compelled to do so.

Same thing with CSV, filesystem access, DB api, etc. They're not the best libraries at the time of any script you're writing, but the reality is that you never really need the best, most ergonomic library ever to get you through a task.

Because of this, many big complex packages like Django have hardly any external dependencies.

If anything you're not the one getting stuck with date APIs; it's the Python core devs. Maintainers of other packages are always free to choose other dependencies, but they almost invariably find that the Python stdlib is good enough for everything.

Re: Rust’s dependencies are starting to worry me

#183

Post by philsnow »

> do I even need this crate at all? 35 lines later I had the parts of dotenv I needed.

I'm not saying you copy-pasted those 35 lines from dotenvy, but for the sake of argument let's say you did: now you can't automatically benefit from dotenvy patching some security issue in those lines.

Re: Rust’s dependencies are starting to worry me

#184

Post by nicoburns »

poincaredisk wrote: post #142
nicoburns wrote: post #90

Earlier quoted context omitted.

As far as I'm aware, LTO completely solves this from a binary size perspective. It will optimise out anything unused. You can still get hit from a build time perspective though.

"completely solves" is a bit of an overstatement. Imagine a curl-like library that allows you to make requests by URL. You may only ever use HTTP urls, but code for all the other schemas (like HTTPS, FTP, Gopher) needs to be compiled in as well. This is an extreme example, but the same thing happens very often at a smaller scale. Optional functionality can't always be removed statically.

I guess that depends on the implementation. If you're calling through an API that dynamically selects the protocol than I guess it wouldn't be removable.

Rust does have a feature flagging system for this kind of optional functionality though. It's not perfect, but it would work very well for something like curl protocol backends though.

Re: Rust’s dependencies are starting to worry me

#185

Post by dathinab »

amelius wrote: post #154

I think the main problem is that you should be able to run dependencies inside their own sandbox, and the language focuses only on memory safety within a monolithic program.

the problem is if you put library dependencies in their own sandbox you have a different kind of interface (much more limited) for libraries

like e.g. if we look at sandbox boundaries we have:

- some in language permission enforcement (e.g. Java Security Manage) -- this approach turned out to be a very bad idea

- process boundaries, i.e. take the boundary the OS enforces and lock it down more (e.g. by stuff like pledge, cgroups etc.) -- this approach turned out okayish

- VM boundaries (e.g. firecracker VMs) -- tourned out well

- emulation boundaries (e.g. WASM) -- mixed history, can turn out well especially if combined with worker processes which lock themself down

but what that means in practice is that wanting the reliably sand box library dependencies will most likely lead to more or less IPC boundaries between the caller and the libary

what that means is practice it's unsuited for a lot of thing

e.g. for most utility lib it's very unsuited

e.g. for a lot (but not all) data structure libs its unsuited and might be a huge issue

e.g. you can apply it to a web-server, but then you are basically reinventing CGI, AGI which okay but can quite compete with perf.

e.g. but you can't apply it to some fundamental runtime engine (e.g. tokio), worse you now might have one copy of the engine running per sandbox... (but you can apply it to some sub-part of tokio internals)

People have tried this a lot in various ways.

But so far this always died off in the long run.

Would be nice if the latest push based around WASM would have some long term success.

Re: Rust’s dependencies are starting to worry me

#187

Post by RossBencina »

kdps wrote: post #147
zaptheimpaler wrote: post #111

This is just a modern problem in all software development, regardless of language. We are doing more complex things, we have a much bigger library of existing code to draw from and there are many reasons to use it. Ultimately a dependency is untrusted code, and there's a long road to go in hardening entire systems to make running arbitrary dependencies safe (if its even possible). In the absence of a technical soluti…

I'd argue that the severity varies between languages, despite the core problem being universal. Languages with comprehensive standard libraries have an advantage over those with minimal built-in functionality, where people rely on external dependencies even for the most basic things (e.g. see Java/.NET vs JS/Node). Lightweight is not always better.

> Languages with comprehensive standard libraries have an advantage

I don't see the advantage. Just a different axis of disadvantage. Take python for example. It has a crazy big standard library full of stuff I will never use. Some people want C++ to go in that direction too -- even though developers are fully capable of rolling their own. Similar problem with kitchen-sink libraries like Qt. "batteries included" languages lead to higher maintenance burden for the core team, and hence various costs that all users pay: dollars, slow evolution, design overhead, use of lowest common denominator non-specialised implementations, loss of core mission focus, etc.

Re: Rust’s dependencies are starting to worry me

#188

Post by tialaramex »

harha_ wrote: post #121
zaptheimpaler wrote: post #111

This is just a modern problem in all software development, regardless of language. We are doing more complex things, we have a much bigger library of existing code to draw from and there are many reasons to use it. Ultimately a dependency is untrusted code, and there's a long road to go in hardening entire systems to make running arbitrary dependencies safe (if its even possible). In the absence of a technical soluti…

Regardless of language, really? I highly doubt that, you don't generally see such problems with C or even C++ because dependencies are more cumbersome to add, especially in a way that's cross-platform.

With C++ it's hilarious because the C++ community is so allergic to proper dependency management and also so desperate for stuff from third party libraries that the committee spends large amounts of its time basically doing dependency management for the community by baking in large features you'd ordinarily take as a dependency into the mandatory standard library.

I'm sure I'll miss some, but IIRC C++ 26 is getting the entire BLAS, two distinct delayed reclamation systems and all of the accompanying infrastructure, new container types, and a very complicated universal system of units.

All of these things are cool, but it's doubtful whether any of them could make sense in a standard library, however for C++ programers that's the easiest way to use them...

It's bedlam in there and of course the same C++ programmers who claim to be "worried" that maybe somebody hid something awful in Rust's crates.io are magically unconcerned that copy-pasting tens of millions of lines of untested code from a third party into absolutely every C++ program to be written in the future could be a bad idea.

Re: Rust’s dependencies are starting to worry me

#189

Post by vvanders »

poincaredisk wrote: post #142
nicoburns wrote: post #90

Earlier quoted context omitted.

As far as I'm aware, LTO completely solves this from a binary size perspective. It will optimise out anything unused. You can still get hit from a build time perspective though.

"completely solves" is a bit of an overstatement. Imagine a curl-like library that allows you to make requests by URL. You may only ever use HTTP urls, but code for all the other schemas (like HTTPS, FTP, Gopher) needs to be compiled in as well. This is an extreme example, but the same thing happens very often at a smaller scale. Optional functionality can't always be removed statically.

That only applies when dynamic dispatch is involved and the linker can't trace the calls. For direct calls and generics(which idiomatic Rust code tends to prefer over dyn traits) LTO will prune extensively.

Re: Rust’s dependencies are starting to worry me

#190

Post by dathinab »

munificent wrote: post #116
MeetingsBrowser wrote: post #77

Earlier quoted context omitted.

I agree that relying on unknown dependencies is a risk, but this misses the point IMO. Number of dependencies and disk space are kind of arbitrary. > Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight. The lightest weight javascript program relies on V8 to run, which has multiple orders of magnitude more dependencies. Most of which you have never heard of. At least cargo makes it…

> relies on V8 to run, which has multiple orders of magnitude more dependencies. Actually, this isn't true. (Or at least wasn't a while back.) I used to work with a bunch of ex-V8 folks and they really despised third-party dependencies and didn't trust any code they didn't write. They used a few third-party libs but for them most part, they tried to own everything themselves.

they are also Google

.. as in they can afford to rewrite everything

.. can afford to suffer from not invented here syndrome

.. and are under _massive_ threat of people doing supply chain attacks compared to most other projects (as they end up running on nearly any desktop computer and half the phones out there)

this just isn't viable for most projects, not just resource/time investment wise, but also reinventing/writing everything isn't exactly good to reduce bugs if you haven't to reliably access to both resources _and_ expertise. Most companies have to live with having many very average developers, and very tight resource limits.

Post reply on HN