Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust’s dependencies are starting to worry me

vincents.dev

Post reply on HN
Print view View original

481–490 of 593 posts

Re: Rust’s dependencies are starting to worry me

#481

Post by andybak »

lcnPylGDnU4H9OF wrote: post #445
andybak wrote: post #442

Earlier quoted context omitted.

> Web server" is, more or less, about converting a database into JSON and/or HTML You seem to have a very different definition of "web server" to me.

Just to explain this confusion, the term “web server” typically refers specifically to software that is listening for HTTP requests, such as apache or nginx. I would use the term “application server” to refer to the process that is processing requests that the web server sends to it. I read “web server” in their comment as “application server” and it makes sense.

Yes. That's the same distinction I would expect. Although I'm not sure that the database stuff is the role I'd usually look for in the application server itself.

Maybe it's a language community thing.

Re: Rust’s dependencies are starting to worry me

#482

Post by constantcrying »

hu3 wrote: post #471
constantcrying wrote: post #413

Earlier quoted context omitted.

Why are you complaining that a project you do not care about is using 13 dependencies, all of which, to your knowledge, are absolutely essential for the functionality? >There is a project that's 500kb in code with 120 dependencies And therefore some project using 13 dependencies is doing it wrong? What are you on about. Obviously there is an enormous abuse of dependencies in the JS ecosystem, who cares?

Their original complaint was about the project taking 20GB of disk space to compile. Also they did point out that the parser depends on a serialisation library, so you're also mistaken about parent thinking the dependencies are necessary. On another note, this pervasive kind of passive aggressive, hand-wavy, tribalistic, blind defense of certain technologies speak volumes about their audiences.

Please actually read the conversation before commenting.

Re: Rust’s dependencies are starting to worry me

#483

Post by bsaul »

wizzwizz4 wrote: post #440
bsaul wrote: post #344

Earlier quoted context omitted.

Random remark: I've noticed the quality of rust libraries too. Which made me really surprised to see the overengineered mess that is the async-openai crate. how can one take an api as simple as openai's one, and turn it to this steaming pile of manure ? in the end, i used reqwest and created my queries manually. I guess that's what everyone does...

The kinds of people who think OpenAI's tech is worth touching with a bargepole are generally not the kinds of people who develop and maintain high-quality Rust libraries.

i get it, but openai being the hotest stuff happening in software for the past.. x years , i would have assumed there was some kind of official client correctly maintained for rust.

I was a bit shocked to be honest.

edit : i originally misread your comment. OpenAI is an important tech, no matter what you think of the company itself. Being able to easily interface with their api is important.

Re: Rust’s dependencies are starting to worry me

#484

Post by galangalalgol »

sseagull wrote: post #458
PoignardAzur wrote: post #388

Earlier quoted context omitted.

> At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number. I'm not convinced that happens that often. As someone working on a Rust library with a fairly heavy depende…

Not in Rust, but I've seen it with Python in scientific computing. Someone needs to do some minor matrix math, so they install numpy. Numpy isn't so bad, but if installing it via conda it pulls in MKL, which sits at 171MB right now (although I have memories of it being bigger in the past). It also pulls in intel-openmp, which is 17MB. Just so you can multiply matrices or something.

Because rust is paying the price to compile everything fromch scratch on a release build, you can pay a little extra to turn on link time optimization and turn of parallelism on release builds and absolutely nothing gets compiled in that you don't use, and nothing gets repeated. Also enabling symbols to be stripped can take something with tokio, clap, serde, nalgebra (matrix stuff) and still be 2-5Mb binary. That is still huge to me because I'm old, but you can get it smaller if you want to recompile std along with your other dependencies.

Re: Rust’s dependencies are starting to worry me

#485

Post by fn-mote »

sseagull wrote: post #458
PoignardAzur wrote: post #388

Earlier quoted context omitted.

> At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number. I'm not convinced that happens that often. As someone working on a Rust library with a fairly heavy depende…

Not in Rust, but I've seen it with Python in scientific computing. Someone needs to do some minor matrix math, so they install numpy. Numpy isn't so bad, but if installing it via conda it pulls in MKL, which sits at 171MB right now (although I have memories of it being bigger in the past). It also pulls in intel-openmp, which is 17MB. Just so you can multiply matrices or something.

> Someone needs to do some minor matrix math, so they install numpy

I’m just not convinced that it’s worth the pain to avoid installing these packages.

You want speedy matrix math. Why would you install some second rate package just because it has a lighter footprint on disk? I want my dependencies rock solid so I don’t have to screw with debugging them. They’re not my core business - if (when) they don’t “just work” it’s a massive time sink.

NumPy isn’t “left pad” so this argument doesn’t seem strong to me.

Re: Rust’s dependencies are starting to worry me

#486

Post by kibwen »

buzzerbetrayed wrote: post #467
kibwen wrote: post #448

Earlier quoted context omitted.

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

Finding a single library that avoids the problem is pretty useless. You can find great libraries in Node as well but everyone would agree that Node has a dependency problem.

And yet it's telling that, when the author mused about library quality and unknowingly suggested an arbitrary library as an example, the Rust version turned out to be high quality.

Re: Rust’s dependencies are starting to worry me

#487

Post by kibwen »

hombre_fatal wrote: post #455
kibwen wrote: post #448

Earlier quoted context omitted.

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

Which glob crate? https://crates.io/search?q=glob I went to page 8 and there were still glob libraries.

The one that shows up first, which is to say, the one with 200 million downloads, which is to say, the one whose name is the exact match for the search query.

Re: Rust’s dependencies are starting to worry me

#488

Post by kibwen »

hu3 wrote: post #453
kibwen wrote: post #448

Earlier quoted context omitted.

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

glob was just an example. They weren't asking about a specific crate. Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. A bad example all around.

> Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving.

To reiterate, lots of things that people in this thread are asking the language to provide are in fact provided by the rust-lang organization: regex, serde, etc. The goalposts are retreating over the horizon.

Rust's primary sin here is that it makes dependency usage transparent to the end-user. Nobody wants to think about how many libraries they depend upon and how many faceless people it takes to maintain those libraries, so they're uncomfortable when Rust shows you. This isn't a Rust problem, it's a software complexity problem.

Re: Rust’s dependencies are starting to worry me

#489

Post by mplanchard »

whstl wrote: post #470
okanat wrote: post #406

Earlier quoted context omitted.

Having some of those libraries listed and then not being able to change API or the implementation is what killed modern C++ adoption (along with the language being a patchwork on top of C). As some of the previous commenters said, when you focus your language to make it easy to write a specific type of program, then you make tradeoffs that can trap you in those constraints like having a runtime, a garbage collector a…

> Being able to swap out the regex, datetime, arg parsing and encoding are a feature A feature present on every language that has those in the stdlib.

Not necessarily, when other components of the stdlib depend on them

Re: Rust’s dependencies are starting to worry me

#490

Post by kibwen »

blub wrote: post #308
tialaramex wrote: post #188

Earlier quoted context omitted.

With C++ it's hilarious because the C++ community is so allergic to proper dependency management and also so desperate for stuff from third party libraries that the committee spends large amounts of its time basically doing dependency management for the community by baking in large features you'd ordinarily take as a dependency into the mandatory standard library. I'm sure I'll miss some, but IIRC C++ 26 is getting t…

If something is in the standard library, then it’s written and vetted by the standard library provider, not by a random third party like you make it sound. With Rust, it’s literally a random third party.

> With Rust, it’s literally a random third party.

No, tons of the foundational Rust crates that show up in every dependency tree are first-party crates provided by the Rust project itself.

Post reply on HN