Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

291–300 of 593 posts

Re: Rust’s dependencies are starting to worry me

#291

Post by cmrdporcupine »

ArthurStacks wrote: post #273
cmrdporcupine wrote: post #229

Earlier quoted context omitted.

This is the default way of doing things in the monorepo(s) at Google. It feels like torture until you see the benefits, and the opposite ... the tangled mess of multiple versions and giant transitive dependency chains... agony. I would prefer to work in shops that manage their dependencies this way. It's hard to find.

[flagged]

It absolutely is so, or was for the 10 years I was there. I worked on Google3 (in Ads, on Google WiFi, on Fiber, and other stuff), in Chromium/Chromecast, Fiber, and on Stadia, and every single one of those repos -- all different repositories -- used vendored deps.

Re: Rust’s dependencies are starting to worry me

#292

Post by [deleted] »

Avi-D-coder wrote: post #290

The solution is strong compile time and runtime guarantees about code behavior. The author is right there's no way an individual can audit all that code. Currently all that code can run arbitrary build code at compile time on the devs machine, it can also run arbitrary unsafe code at runtime, make system calls, etc.. Software is not getting simpler, the abundance of high quality libraries is great for Rust, but there…

[deleted]

Re: Rust’s dependencies are starting to worry me

#293

Post by [deleted] »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

[deleted]

Re: Rust’s dependencies are starting to worry me

#294

Post by arp242 »

righthand wrote: post #47

Everyone is in such a rush to get their project out the door, no one has time to generate a key and properly code sign releases and begin developing a more secure chain. Now we have JS package "whatever code" ecosystem but for Rust. As if we haven't watched NPM get hacked many times over the last decade or so.

> As if we haven't watched NPM get hacked many times over the last decade or so.

When has this happened? The only one I remember is the event-stream thing, and that was what, over five years ago? Doesn't seem all that common from what I can see?

Re: Rust’s dependencies are starting to worry me

#295

Post by eviks »

EasyMark wrote: post #241
rs186 wrote: post #71

I once wanted to contribute to the popular swc project ( https://github.com/swc-project/swc ). I cloned the repo, ran build, and a whooping 20GB was gone from my disk. The parser itself ( https://github.com/swc-project/swc/blob/main/crates/swc_ecma... ) has over a dozen dependencies, including serde. Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight. I decided that I should leave…

I just built it with "git clone --depth 1 ..." and the build from cargo build --release is 2.9GB (2.3GB in the target folder)?

You should also add the dev build

Re: Rust’s dependencies are starting to worry me

#296

Post by deepsun »

ardit33 wrote: post #249
sitkack wrote: post #224

Earlier quoted context omitted.

Way back when, I used to vendor all the libraries for a project (Java/Cpp/Python) into a mono repo and integrate building everything into the projects build files so anyone could rebuild the entire app stack with whatever compiler flags they wanted. It worked great, but it took diligence, it also forces you to interact with your deps in ways that adding a line to a deps file does not.

Maven was the one the started the downfall into dependency hell. (Ant as well, but it was harder to blindly include things into it) Kids today don't know how to do that anymore...

Yet Maven repository is still not that bloated even after 20+ years Java et al. being one of the most popular language.

Compared to Rust where my experience with protobuf lib some time ago was that there is a choice of not 1 but even 3 different libraries, one of which doesn't support services, another didn't support the syntax we had to support, and the third one was unmaintained. So out of 3 choices no single one worked.

Compared that to Maven, where you have only one official supported choice that works well and well maintained.

Re: Rust’s dependencies are starting to worry me

#297

Post by hinkley »

kazinator wrote: post #222
socalgal2 wrote: post #214

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a li…

> Should a 'glob' library actually read the file system and give you filenames The POSIX glob function after which these things are named traverses the filesystem and matches directory entries. The pure matching function which matches a glob pattern against a filename-like string is fnmatch . But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Nobody should be trying…

They eventually fixed it but grunt once upon a time used a glob implementation that could not short circuit on wildcards in ignore patterns. So I caught it scanning the node-modules directory and then dropping every file it found because it matched on “node_modules/**”. Builds got a lot faster when I pushed that update out.

There’s a lot of stupid ways to implement glob and only a couple of smart ones.

Re: Rust’s dependencies are starting to worry me

#298

Post by scripturial »

zahlman wrote: post #274
kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

> The only real solution I can think of to deal with this long term is ultra-fine-grained symbols and dependencies. Every function, type, and other top-level language construct needs to declare the set of things it needs to run (other functions, symbols, types, etc). When you depend on that one symbol it can construct, on demand, the exact graph of symbols it needs and dump the rest for any given library. You end up…

If you think about it, every function already declares what it needs simply by actually using it. You know if a function needs another function because it calls it. So what exactly are you asking? That the programmer insert a list of dependent functions in a comment above every function? The compiler could do that for you. The compiler could help you and go up a level and insert the names of modules the functions belong to?

Re: Rust’s dependencies are starting to worry me

#299

Post by boguscoder »

ardit33 wrote: post #249
sitkack wrote: post #224

Earlier quoted context omitted.

Way back when, I used to vendor all the libraries for a project (Java/Cpp/Python) into a mono repo and integrate building everything into the projects build files so anyone could rebuild the entire app stack with whatever compiler flags they wanted. It worked great, but it took diligence, it also forces you to interact with your deps in ways that adding a line to a deps file does not.

Maven was the one the started the downfall into dependency hell. (Ant as well, but it was harder to blindly include things into it) Kids today don't know how to do that anymore...

This works very well until different parts of the deps tree start pulling same Foo with slightly different flags/settings. Often for wrong reasons but sometimes for right ones, and then its new kind of “fun”. Sometimes buildsystem is there to help you but sometimes you are on your own. Native languages like C++ bring special kind of joy called ODR violations to the mix…

Re: Rust’s dependencies are starting to worry me

#300

Post by iTokio »

WuxiFingerHold wrote: post #271
kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

> IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. Go and C# (.NET) are counterexamples. They both have great ecosystems and just as simple and effective package management as Rust or JS (Node). But neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs a…

I think this is partially true, but more nuanced than just saying that Rust std lib is lacking.

Compared to go and c#, Rust std lib is mostly lacking:

- a powerful http lib

- serialization

But Rust approach, no Runtime, no GC, no Reflection, is making it very hard to provide those libraries.

Within these constraints, some high quality solutions emerged, Tokio, Serde. But they pioneered some novel approaches which would have been hard to try in the std lib.

The whole async ecosystem still has a beta vibe, giving the feeling of programming in a different language. Procedural macros are often synonymous with slow compile times and code bloat.

But what we gained, is less runtime errors, more efficiency, a more robust language.

TLDR: trade-offs everywhere, it is unfair to compare to Go/C# as they are languages with a different set of constraints.

Post reply on HN