Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

221–230 of 593 posts

Re: Rust’s dependencies are starting to worry me

#221

Post by mleonhard »

jerf wrote: post #29

A true enough statement, but "Rust" is unnecessarily specific. Dependencies are getting scary in general. Supply chain attacks are no longer hypothetical, they're here and have been for a while. If I were designing a new language I think I'd be very interested in putting some sort of capability system in so I can confine entire library trees safely, and libraries can volunteer somehow what capabilities they need/offe…

I love this idea and I hope I get to work on it someday. I've wanted this ever since I was a starry-eyed teenager on IRC listening to Darius Bacon explain his capability-based OS idea, aptly called "Vapor".

I think it could be possible in Rust with a linter, something like https://github.com/geiger-rs/cargo-geiger . The Rust compiler has some unsoundness issues such as https://github.com/rust-lang/rust/issues/84366 . Those would need fixing or linter coverage.

Re: Rust’s dependencies are starting to worry me

#222

Post by kazinator »

socalgal2 wrote: post #214

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a li…

> Should a 'glob' library actually read the file system and give you filenames

The POSIX glob function after which these things are named traverses the filesystem and matches directory entries.

The pure matching function which matches a glob pattern against a filename-like string is fnmatch.

But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob.

Nobody should be trying to implement glob from scratch using a fnmatch-like function and directory traversal. It is not so trivial.

glob performs a traversal that is guided by the pattern. It has to break the pattern into path components. It knows that "*/*/*" has three components and so the traversal will only go three levels deep. Also "dir/*" has a component which is a fixed match, and so it just has to open "dir" without scanning the current directory; if that fails, glob has failed.

If the double star ** is supported which matches multiple components, that's also best if it likewise integrated into glob.

If brace expansion is supported, that adds another difficulty because different branches of a brace can have different numbers of components, like {*/x,*/*/x,*/*/*/x}. To implement glob, it would greatly help us to have brace expansion as a separate function which expands the braces, producing multiple glob patterns, which we can then break into path components and traverse.

Re: Rust’s dependencies are starting to worry me

#223

Post by sitkack »

the__alchemist wrote: post #199

To address a point near the end of the article, here is my [partial] solution that works as a baseline. Curate a collection of libraries you use and trust. This will probably involve making a number of your own. Wheel-reinvention, if you will. If done properly, even the upfront time cost will save in the long-run. I am in the minority here, but I roll my own libs whenever possible, and the 3rd party libs I use are of…

Rust really made some unfortunate choices with async, it pollutes everything but isn't generic enough so now you are married to the runtime, this bifurcates the whole ecosystem. It is nearly phobos/demios problem from Dlang, but instead Tokio just took over. One doesn't use Rust anymore, they use Tokio.

Re: Rust’s dependencies are starting to worry me

#224

Post by sitkack »

kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

Way back when, I used to vendor all the libraries for a project (Java/Cpp/Python) into a mono repo and integrate building everything into the projects build files so anyone could rebuild the entire app stack with whatever compiler flags they wanted.

It worked great, but it took diligence, it also forces you to interact with your deps in ways that adding a line to a deps file does not.

Re: Rust’s dependencies are starting to worry me

#225

Post by tough »

socalgal2 wrote: post #214

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a li…

Bun has a glob too https://bun.sh/docs/api/glob

Re: Rust’s dependencies are starting to worry me

#226

Post by f_devd »

rafram wrote: post #197
vvanders wrote: post #189

Earlier quoted context omitted.

That only applies when dynamic dispatch is involved and the linker can't trace the calls. For direct calls and generics(which idiomatic Rust code tends to prefer over dyn traits) LTO will prune extensively.

let uri = get_uri_from_stdin(); networking_library::make_request(uri); How is the compiler supposed to prune that?

  let uri: Uri = get_uri_from_stdin().parse()?;
If the library is made in a modular way this is how it would typically be done. The `HTTP` may be inferred by calls further along in the function.

Re: Rust’s dependencies are starting to worry me

#227

Post by elcritch »

eddd-ddde wrote: post #68
neilv wrote: post #19

In the past (not in Rust, but other languages), for important systems, I've instituted policies of minimizing dependencies from these language-specific package repositories, and for the ones you do use, having to copy it to our own repos and audit each update before use. But that's not practical for all situations. For example, Web frontend developer culture might be the worst environment, to the point you often can'…

The cool thing about rust is you can implement async yourself. You aren't tied to any specific implementation.

Except that libraries using different async libraries in Rust seem generally incompatible.

Re: Rust’s dependencies are starting to worry me

#228

Post by foresto »

nicoburns wrote: post #90
kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

As far as I'm aware, LTO completely solves this from a binary size perspective. It will optimise out anything unused. You can still get hit from a build time perspective though.

> As far as I'm aware, LTO completely solves this from a binary size perspective.

I wouldn't say completely. People still sometimes struggle to get this to work well.

Recent example: (Go Qt bindings)

https://github.com/mappu/miqt/issues/147

Re: Rust’s dependencies are starting to worry me

#229

Post by cmrdporcupine »

sitkack wrote: post #224
kion wrote: post #85

IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. That's essentially where we are today both in language repositories for OSS languages and private monorepos. This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painsta…

Way back when, I used to vendor all the libraries for a project (Java/Cpp/Python) into a mono repo and integrate building everything into the projects build files so anyone could rebuild the entire app stack with whatever compiler flags they wanted. It worked great, but it took diligence, it also forces you to interact with your deps in ways that adding a line to a deps file does not.

This is the default way of doing things in the monorepo(s) at Google.

It feels like torture until you see the benefits, and the opposite ... the tangled mess of multiple versions and giant transitive dependency chains... agony.

I would prefer to work in shops that manage their dependencies this way. It's hard to find.

Re: Rust’s dependencies are starting to worry me

#230

Post by EasyMark »

pjmlp wrote: post #135
MeetingsBrowser wrote: post #79

Earlier quoted context omitted.

> Devs can add whatever they feel like on their workstations A compromised dev machine is also a problem.

True, hence we can go next level and also deal with limited accounts for developers, and I can tell you most folks on HN would hate to work in such corporate environments.

I'd leave. If I have to beg IT security every other day for something,it's just not worth it. I was in that situation once before and it was endlessly frustrating. It also wasn't even their choice, the CEO dictated it after attending some security talk once upon a time, and then instantly "you can't trust anyone or anything". You can trust my stay there will be short though :)
Post reply on HN