Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

441–450 of 593 posts

Re: Rust’s dependencies are starting to worry me

#441

Post by andybak »

nullc wrote: post #421
PoignardAzur wrote: post #388

Earlier quoted context omitted.

> At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number. I'm not convinced that happens that often. As someone working on a Rust library with a fairly heavy depende…

I was very 'impressed' to see multiple SSL libraries pulled into rust software that never makes a network connection.

This is where a) a strong stdlib and b) community consensus on common packages tends to help at least mitigate the problem.

My feeling is that Python scores fairly well in this regard. At least it used to. I haven't been following closely in recent years.

Re: Rust’s dependencies are starting to worry me

#442

Post by andybak »

mort96 wrote: post #423
dgb23 wrote: post #380

Earlier quoted context omitted.

“Web server” is a pretty big use case though. But I agree that graphics is often overlooked in std libs. However that’s a bit of a different beast. Std libs typically deal with what the OS provides. Graphics is its own world so to speak. As for Wasm: first, that’s a runtime issue and not a language issue. I think GC is on the roadmap for Wasm. Second, Go and C# obviously predate Wasm. In the end, not every language s…

"Web server" is, more or less, about converting a database into JSON and/or HTML. There are complexities there, sure, but it's not like it's some uniquely monumental undertaking compared to other fields.

> Web server" is, more or less, about converting a database into JSON and/or HTML

You seem to have a very different definition of "web server" to me.

Re: Rust’s dependencies are starting to worry me

#443

Post by djfivyvusn »

fiedzia wrote: post #435
WuxiFingerHold wrote: post #271

Earlier quoted context omitted.

> IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. Go and C# (.NET) are counterexamples. They both have great ecosystems and just as simple and effective package management as Rust or JS (Node). But neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs a…

> but neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs They also have a lot narrower scope of use, which means it is easier to create stdlib usable for most people. You can't do it with more generic language.

I didn't understand the embedded systems argument. Just because a standard lib is large doesn't mean it all ends up in the compilation target.

Re: Rust’s dependencies are starting to worry me

#444

Post by arccy »

ak_111 wrote: post #422
espoal wrote: post #414

Earlier quoted context omitted.

Why the compiler can't detect it will not be used? Tree shaking is well implemented in Javascript compilers, an ecosystem which extensively suffer from this problem. It should be possible to build a dependency graph and analyze which functions might actually end up in the scope. After all the same is already done for closures.

As poster dead deep in the thread below, something like this can happen doc_format = get_user_input() parsed_doc = foolib.parse(doc_format) You as the implementer might know the user will never input xml, so doc_format can't be 'xml' (you might even add some error handling if the user inputs this), but how can you communicate this to the compiler?

That's called bad library design. Rather than a global, make an instantiated parser that takes in specific codecs.

Re: Rust’s dependencies are starting to worry me

#445

Post by lcnPylGDnU4H9OF »

andybak wrote: post #442
mort96 wrote: post #423

Earlier quoted context omitted.

"Web server" is, more or less, about converting a database into JSON and/or HTML. There are complexities there, sure, but it's not like it's some uniquely monumental undertaking compared to other fields.

> Web server" is, more or less, about converting a database into JSON and/or HTML You seem to have a very different definition of "web server" to me.

Just to explain this confusion, the term “web server” typically refers specifically to software that is listening for HTTP requests, such as apache or nginx. I would use the term “application server” to refer to the process that is processing requests that the web server sends to it. I read “web server” in their comment as “application server” and it makes sense.

Re: Rust’s dependencies are starting to worry me

#447

Post by Measter »

espoal wrote: post #414
ak_111 wrote: post #372

Earlier quoted context omitted.

"dependency" here I guess means something higher-level that your compiler can't make the assumption you will never use. For example you know you will never use one of the main functions in the parsing library with one of the arguments set to "XML", because you know for sure you don't use XML in your domain (for example you have a solid project constraint that says XML is out of scope). Unfortunately the code dealing…

Why the compiler can't detect it will not be used? Tree shaking is well implemented in Javascript compilers, an ecosystem which extensively suffer from this problem. It should be possible to build a dependency graph and analyze which functions might actually end up in the scope. After all the same is already done for closures.

What you're calling "tree shaking" is more commonly called "dead code elimination" in compilers, and is one of the basic optimisations that any production compiler would implement.

Re: Rust’s dependencies are starting to worry me

#448

Post by kibwen »

socalgal2 wrote: post #214

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a li…

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob

213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

Re: Rust’s dependencies are starting to worry me

#449

Post by Xeoncross »

athrowaway3z wrote: post #301
WuxiFingerHold wrote: post #271

Earlier quoted context omitted.

> IMO any system where taking a dependency is "easy" and there is no penalty for size or cost is going to eventually lead to a dependency problem. Go and C# (.NET) are counterexamples. They both have great ecosystems and just as simple and effective package management as Rust or JS (Node). But neither Go or C# have issues with dependency hell like Rust or even more JavaScript, because they have exceptional std libs a…

No it doesn't. A large stdlib solves the problems the language is focused on. For C# and Go that is web hosts. Try using them outside that scope and the dependencies start to pile in (Games, Desktop) or they are essentially unused (embedded, phones, wasm)

> A large stdlib solves the problems the language is focused on

That's part of it, but it also solves the problem of vetting. When I use a Go stdlib I don't have to personally spend time to vet it like it do when looking at a crate or npm package.

In general, Go & Rust packages on github are high quality to begin with, but there is still a pronounced difference between OS packages and what is approved to be part of the language's own stdlib.

It's nice to know thousands of different companies already found the issues for me or objected to them in reviews before the library was published.

Re: Rust’s dependencies are starting to worry me

#450

Post by zahlman »

voidnap wrote: post #353
notpushkin wrote: post #306

Earlier quoted context omitted.

There are things added from tine to time, but yeah, some stuff in there just feels dated at this point. I’m still hoping we can get a decently typed argparse with a modern API though (so much better for tiny scripts without deps!)

I'm thankful argparse exists in pythons stdlib. But argument parsing is not that hard especially for simpler programs. programmers should be able to think for a minute and figure it out instead of always reaching for clap, thats how you get dependency hell. Argument parsing, in partucular, is a great place to start realizing that you can implement what you need without adding a dozen dependencies

Argument parsing is absolutely the kind of thing where I'd reach for a third-party library if the standard library didn't provide (and in Python's case, maybe even then - argparse has some really unpleasant behaviours). When you look through library code, it might seem like way more than you'd write yourself, and it probably is. But on a conceptual level you'll probably actually end up using a big chunk of it, or at least see a future use for it. And it doesn't tend to pull in a lot of dependencies. (For example, click only needs colorama, and then only on Windows; and that doesn't appear to bring in anything transitively.)

It's a very different story with heavyweight dependencies like Numpy (which include reams of tests, documentation and headers even in the wheels that people are only installing to be a dependency of something else, and covers a truly massive range of functionality including exposing BLAS and LAPACK for people who might just want to multiply some small matrices or efficiently represent an image bitmap), or the more complex ones that end up bringing in multiple things completely unrelated to your project that will never be touched at runtime. (Rich supports a ton of wide-ranging things people might want to do with text in a terminal, and I would guess most clients probably want to do exactly one of those things.)

Post reply on HN