Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

191–200 of 593 posts

Re: Rust’s dependencies are starting to worry me

#191

Post by ATechGuy »

dathinab wrote: post #185
amelius wrote: post #154

I think the main problem is that you should be able to run dependencies inside their own sandbox, and the language focuses only on memory safety within a monolithic program.

the problem is if you put library dependencies in their own sandbox you have a different kind of interface (much more limited) for libraries like e.g. if we look at sandbox boundaries we have: - some in language permission enforcement (e.g. Java Security Manage) -- this approach turned out to be a very bad idea - process boundaries, i.e. take the boundary the OS enforces and lock it down more (e.g. by stuff like pled…

Thanks! This is a very detailed explanation of why existing sandboxing techniques will not work as expected for dependencies (wrt to functionality or performance).

Re: Rust’s dependencies are starting to worry me

#192

Post by dezgeg »

pjmlp wrote: post #60
echelon wrote: post #56

Earlier quoted context omitted.

I develop for Linux, Mac, and Windows. Multiple architectures and OSes. I rarely see platform issues with Rust. It's typically only stuff at the edge, like CUDA libraries, that trip up cross-platform builds. Rust, as a systems language, is quite good at working on a variety of systems.

Starts already that Rust won't support architectures not available on LLVM, but on GCC, otherwise having a Rust frontend project for GCC wouldn't be a thing. And the systems language remark, I am still looking forward when sorting ABI issues for binary libraries is finally something that doesn't need to go through solutions designed for C and C++.

What architectures that are missing from LLVM are commercially relevant today and not on well-earned retirement?

Re: Rust’s dependencies are starting to worry me

#193

Post by jampekka »

ChocolateGod wrote: post #173
skydhash wrote: post #88

Earlier quoted context omitted.

From my point of view, the issue stems from developers wanting to control distribution. Fine if it's for your own usage, not really if you're planning for others to use it. You will find the most convoluted build system just because they have a pet platform they want to specially support making it hell to do anything on others. It could be better, but the current solutions (npm, go, python,...) favor only the develop…

There's examples of maintainers/packagers effectively sabotaging other peoples projects when making packages for distros, whether that's shipping them broken, ancient versions etc. e.g. Bottles, WebkitGTK (distros liked keeping this one held back even though doing so is a security risk) IMHO it shouldn't be the responsibility of the OS vendor to package third party applications.

Distro maintainers/packagers are who keep the current software stacks running. It's rather amazing how they manage to keep the billion or so lines of separately written code working in unison.

That said, the labor needed to keep the stuff together could be reduced a lot by the more ergonomical and universal packaging and distribution methods like Cargo (and, dare I say, npm). I think some kind of a better bridge between developers and distros could be found here.

Re: Rust’s dependencies are starting to worry me

#194

Post by James_K »

When I am compiling Rust applications, I must admit I'm always rather bemused at the number of dependencies pulled. Even what I'd have thought to be simple tools reach easily about 200 dependent packages. It's nightmarish. One way this becomes particularly apparent is if you're trying to create a reproducible package for Guix or Nix. You end up having to manually specify a package for every different Rust library because of how those system require reproducible builds. The process of writing Guix package for software has been extremely illuminating for me, as to just how deeply nested certain technologies are vs. others. I'd be willing to bet it's a good metric for what sticks around. If you've got 200 dependencies, I don't think your software is gonna last the test of time. It seems a recipe for endless churn.

Re: Rust’s dependencies are starting to worry me

#195

Post by rcxdude »

timewizard wrote: post #73
nemothekid wrote: post #43

I feel like leftpad has given package managers a very bad name. I understand the OP's hesitation, but it feels a little ridiculous to me. tokio is a work-stealing, asynchronous runtime. This is a feature that would be an entire language . Does OP consider it reasonable to audit the entire Go language? or the V8 engine for Node? v8 is ~10x more lines than tokio. If Cloudflare uses Node, would you expect Cloudflare to…

If two different dependencies use a different version of some other dependency between them does cargo still include both versions by default? This is something I've only ever seen cargo do.

It'll do that if there isn't a single version that meets both requirements. Which is a great thing, because most other languages will just fail the build in that case (well, there are still cases where it won't even work in rust, if types from those sub-dependencies are passed in between the two closer dependencies)

Re: Rust’s dependencies are starting to worry me

#196

Post by tialaramex »

jerf wrote: post #29

A true enough statement, but "Rust" is unnecessarily specific. Dependencies are getting scary in general. Supply chain attacks are no longer hypothetical, they're here and have been for a while. If I were designing a new language I think I'd be very interested in putting some sort of capability system in so I can confine entire library trees safely, and libraries can volunteer somehow what capabilities they need/offe…

You want a special purpose language.

In your particular example of image loading, you want WUFFS. https://github.com/google/wuffs

In WUFFS most programs are impossible. Their "Hello, world" doesn't print hello world because it literally can't do that. It doesn't even have a string type, and it has no idea how to do I/O so that's both elements of the task ruled out. It can however, Wrangle Untrusted File Formats Safely which is its sole purpose.

I believe there should be more special purpose languages like this, as opposed to the General Purpose languages most of us learn. If your work needs six, sixteen or sixty WUFFS libraries to load different image formats, that's all fine because categorically they don't do anything outside their box. Yet, they're extremely fast because since they can't do anything bad by definition they don't need those routine "Better not do anything bad" checks you'd write in a language like C or the compiler would add in a language like Rust, and because they vectorize very nicely.

Re: Rust’s dependencies are starting to worry me

#197

Post by rafram »

vvanders wrote: post #189
poincaredisk wrote: post #142

Earlier quoted context omitted.

"completely solves" is a bit of an overstatement. Imagine a curl-like library that allows you to make requests by URL. You may only ever use HTTP urls, but code for all the other schemas (like HTTPS, FTP, Gopher) needs to be compiled in as well. This is an extreme example, but the same thing happens very often at a smaller scale. Optional functionality can't always be removed statically.

That only applies when dynamic dispatch is involved and the linker can't trace the calls. For direct calls and generics(which idiomatic Rust code tends to prefer over dyn traits) LTO will prune extensively.

    let uri = get_uri_from_stdin();
    networking_library::make_request(uri);
How is the compiler supposed to prune that?

Re: Rust’s dependencies are starting to worry me

#198

Post by crabmusket »

Smaug123 wrote: post #112
0cf8612b2e1e wrote: post #99

Earlier quoted context omitted.

Is there anything in existence which has a version of this idea? It makes a ton of sense to me, but you are right that it would be practically impossible to do in a current language.

Austral, for example? https://austral-lang.org/spec/spec.html#rationale-cap

Austral is a really cool experiment and I love how much effort was put into the spec which you've linked to. It explains the need for capabilities and linear types, and how they interact, really well.

Re: Rust’s dependencies are starting to worry me

#199

Post by the__alchemist »

To address a point near the end of the article, here is my [partial] solution that works as a baseline.

Curate a collection of libraries you use and trust. This will probably involve making a number of your own. Wheel-reinvention, if you will. If done properly, even the upfront time cost will save in the long-run. I am in the minority here, but I roll my own libs whenever possible, and the 3rd party libs I use are often ones I know, have used been for, and vetted that they have a shallow tree of their own.

Is this sustainable? I don't know. But It's the best I've come up with, in order to use what I see as the best programming language available for several domains.

There are a lot of light-weight, excellent libs I will use without hesitation, and have wide suitability. Examples:

  - num_enum
  - num_traits
  - bytemuck
  - chrono
  - rand
  - regex
  - bincode
  - rayon
  - cudarc
Heavier, and periodically experience mutual-version hell, but are are very useful for GUI programs:

  - EGUI
  - WGPU
  - Winit
On a darker note, the rust web ecosystem maybe permanently lost to async and messy dependencies. Embedded is going that way too, but I have more hope there, and am doing my best to have my own tooling.

Re: Rust’s dependencies are starting to worry me

#200

Post by a2128 »

philsnow wrote: post #183

> do I even need this crate at all? 35 lines later I had the parts of dotenv I needed. I'm not saying you copy-pasted those 35 lines from dotenvy, but for the sake of argument let's say you did: now you can't automatically benefit from dotenvy patching some security issue in those lines.

To benefit you have to actually trust the current and future maintainers of the package, its dependencies, the dependencies of its dependencies, etc. You can also automatically get breached in a supply chain attack, so it's a tradeoff
Post reply on HN