Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

271–280 of 593 posts

Re: Rust’s dependencies are starting to worry me

#271

Post by WuxiFingerHold »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

> IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem.

Go and C# (.NET) are counterexamples. They both have great ecosystems and just as simple and effective package management as Rust or JS (Node). But neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs and even large frameworks like ASP.NET or EF Core.

A great std lib is obviously the solution. Some Rust defenders are talking it down by giving Python as counter example. But again, Go and C# are proving them wrong. A great std lib is a solution, but one that comes with huge efforts that can only be made by large organisations like Google (Go) or Microsoft (C#).

Re: Rust’s dependencies are starting to worry me

#272

Post by kgeist »

constantcrying wrote: post #157
demarq wrote: post #5

> Many call for adding more to the rust standard library much like Go This is the way.

That is a serious burden on the maintainers, it creates all kinds of different problems, especially if the functionality of the libraries assumes a certain execution environment. Rust doesn't just target x86 desktops.

Go doesn't just target x86 desktops either

Re: Rust’s dependencies are starting to worry me

#273

Post by ArthurStacks »

cmrdporcupine wrote: post #229
sitkack wrote: post #224

Earlier quoted context omitted.

Way back when, I used to vendor all the libraries for a project (Java/Cpp/Python) into a mono repo and integrate building everything into the projects build files so anyone could rebuild the entire app stack with whatever compiler flags they wanted. It worked great, but it took diligence, it also forces you to interact with your deps in ways that adding a line to a deps file does not.

This is the default way of doing things in the monorepo(s) at Google. It feels like torture until you see the benefits, and the opposite ... the tangled mess of multiple versions and giant transitive dependency chains... agony. I would prefer to work in shops that manage their dependencies this way. It's hard to find.

[flagged]

Re: Rust’s dependencies are starting to worry me

#274

Post by zahlman »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

> The only real solution I can think of to deal with this long term is ultra-fine-grained symbols and dependencies. Every function, type, and other top-level language construct needs to declare the set of things it needs to run (other functions, symbols, types, etc). When you depend on that one symbol it can construct, on demand, the exact graph of symbols it needs and dump the rest for any given library. You end up with the minimal set of code for the functionality you need.

Or you have ultra-fine-grained modules, and rely on existing tree-shaking systems.... ?

Re: Rust’s dependencies are starting to worry me

#275

Post by zahlman »

WuxiFingerHold wrote: post #271
kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

> IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. Go and C# (.NET) are counterexamples. They both have great ecosystems and just as simple and effective package management as Rust or JS (Node). But neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs a…

>A great std lib is obviously the solution. Some Rust defenders are talking it down by giving Python as counter example.

Python's standard library is big. I wouldn't call it great, because Python is over 30 years old and it's hard to add things to a standard library and even harder to remove them.

Re: Rust’s dependencies are starting to worry me

#276

Post by zahlman »

kion wrote: post #269
bilbo-b-baggins wrote: post #266

Earlier quoted context omitted.

Symbol culling and dead code removal is already a thing in modern compilers and linkers, and rust can do it too: https://github.com/johnthagen/min-sized-rust

Others have made similar comments, but tree-shaking, symbol culling and anything else that removes dead code after its already been distributed and/or compiled is too late IMO. It's a band-aid on the problem. A useful and pragmatic band-aid today for sure, but it fundamentally bothers me that we have to spend time compiling code and then spend more time to analyze and rip it back out. Part of the issue I have with th…

> Others have made similar comments, but tree-shaking, symbol culling and anything else that removes dead code after its already been distributed and/or compiled is too late IMO.

Why, in principle, wouldn't the same algorithms work before distribution?

For that matter, check out the `auditwheel` tool in the Python ecosystem.

Re: Rust’s dependencies are starting to worry me

#277

Post by NewJazz »

Out of curiosity I ran toeki a tool for counting lines of code, and found a staggering 3.6 million lines of rust. Removing the vendored packages reduces this to 11136 lines of rust.

Tokei hasn't had a stable release in over 4 years and misreports lines of code in some instances. The author in the past has basically said they would need to be paid to backport one line fixes with no merge conflicts that fix real accuracy issues in their software... Bad look in my book.

https://github.com/XAMPPRocky/tokei/issues/875

Re: Rust’s dependencies are starting to worry me

#278

Post by endorphine »

What actually surprised me in Rust, is the amount of fragmentation and abandoned libraries. For example, serde_yaml is archived and there are two other libraries that do the same (?) thing. It seems like there's a significant effort required to search for and decide which (if at all) library to use. This is not so much pronounced in Go.

Re: Rust’s dependencies are starting to worry me

#279

Post by never_inline »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

Size issues and bloat can be solved by tree shaking which is orthogonal to granularity of the package ecosystem. It doesn't matter for server side (at least people don't care). On client side, most ecosystems have a way to do it. Dart does it. Android does it with proguard.

The more pressing issue with dependencies is supply chain risks including security. That's why larger organizations have approval processes for using anything open source. Unfortunately the new crop of open source projects in JS and even Go seem to suffer from "IDGAF about what shit code from internet I am pulling" syndrome.

Unfortunately granularity does not solve that as long as your 1000 functions come from 1000 authors on NPM.

Re: Rust’s dependencies are starting to worry me

#280

Post by zahlman »

tialaramex wrote: post #188
harha_ wrote: post #121

Earlier quoted context omitted.

Regardless of language, really? I highly doubt that, you don't generally see such problems with C or even C++ because dependencies are more cumbersome to add, especially in a way that's cross-platform.

With C++ it's hilarious because the C++ community is so allergic to proper dependency management and also so desperate for stuff from third party libraries that the committee spends large amounts of its time basically doing dependency management for the community by baking in large features you'd ordinarily take as a dependency into the mandatory standard library. I'm sure I'll miss some, but IIRC C++ 26 is getting t…

> copy-pasting tens of millions of lines of untested code from a third party into absolutely every C++ program to be written in the future could be a bad idea.

Is it really that bad? (By my count, as a point of reference, the Python 3.13 standard library is just under 900k lines for the .py files.)

Post reply on HN