Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can a Rust binary use incompatible versions of the same library?

github.com

Post reply on HN
Print view View original

21–30 of 89 posts

Re: Can a Rust binary use incompatible versions of the same library?

#21

Post by whatshisface »

oefrha wrote: post #16
btilly wrote: post #15

This is great for avoiding conflicts when you try to get your project running. It sucks when there is a vulnerability in a particular library, and you're trying to track all of the ways in which that vulnerable code is being pulled into your project. My preference is to force the conflict up front by saying that you can't import conflicting versions. This creates a constant stream of small problems, but avoids really…

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

Backwards compatibility is more difficult in Rust for many reasons. For example, you can't add a new item to an enum without creating missing-case errors everywhere it is used.

Re: Can a Rust binary use incompatible versions of the same library?

#22

Post by MrJohz »

btilly wrote: post #15

This is great for avoiding conflicts when you try to get your project running. It sucks when there is a vulnerability in a particular library, and you're trying to track all of the ways in which that vulnerable code is being pulled into your project. My preference is to force the conflict up front by saying that you can't import conflicting versions. This creates a constant stream of small problems, but avoids really…

   cargo tree -i log@0.3.9
will show which dependencies require this particular version of log, and how they are transitively related to the main package. In this case, you would clearly see that the out-of-date dependency comes from package "b".

There are equivalents for must other package managers that take this approach, and I've never found this a problem in practice.

Of course, you still need to know that there's a vulnerability there in the first place, but that's why tools like NPM often integrate with vulnerability scanners so that they can check your dependencies as you install them.

Re: Can a Rust binary use incompatible versions of the same library?

#23

Post by dwattttt »

whatshisface wrote: post #21
oefrha wrote: post #16

Earlier quoted context omitted.

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

Backwards compatibility is more difficult in Rust for many reasons. For example, you can't add a new item to an enum without creating missing-case errors everywhere it is used.

That's a true effect, although I'd question whether it makes it harder or easier for things to be backwards compatible. I use Rust because I trust it to throw up a bunch of errors when I make changes; if I handle all the cases of an enum somewhere, and suddenly there's a new enum variant, the answer is probably that I need to handle the new variant there too.

Re: Can a Rust binary use incompatible versions of the same library?

#24

Post by MrJohz »

oefrha wrote: post #16
btilly wrote: post #15

This is great for avoiding conflicts when you try to get your project running. It sucks when there is a vulnerability in a particular library, and you're trying to track all of the ways in which that vulnerable code is being pulled into your project. My preference is to force the conflict up front by saying that you can't import conflicting versions. This creates a constant stream of small problems, but avoids really…

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

But surely you can still run into the same issue:

I import a@1 and b@1 a@1 transitively depends on c@1 b@1 transitively depends on c@2

Even with different import paths, I still have two different versions of c in my codebase. It'll just be that one of them is imported as "c" and the other will be imported as "c/v2" - but you don't need to worry about that, because that's happening in transitive dependencies that you're not writing.

You still have the same issue of needing to keep track of all the different versions that can exist in your codebase.

Re: Can a Rust binary use incompatible versions of the same library?

#25

Post by dwattttt »

tick_tock_tick wrote: post #19
dwattttt wrote: post #13

Earlier quoted context omitted.

Their internal states in Rust are also namespaced, so two incompatible crates in the same process won't observe each others symbols. If they access external resources that are not namespaced though, that could be a problem.

Such as stdout or stderr that a log crate would be using?

That's not a very clear example. You don't need to be using multiple versions of the same dependency to contend on access to stderr/out, just having a println in your code along with logging code will have the same effect.

I haven't ever observed a problem of concurrent access to stdout/err though, I expect because the methods for accessing stdout/err lock them for the duration of their printing. If you Google for "Rust print console slow", you'll probably find advice to explicitly lock it, to avoid individual printlns from each acquiring the lock.

Re: Can a Rust binary use incompatible versions of the same library?

#26

Post by sanxiyn »

whatshisface wrote: post #21
oefrha wrote: post #16

Earlier quoted context omitted.

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

Backwards compatibility is more difficult in Rust for many reasons. For example, you can't add a new item to an enum without creating missing-case errors everywhere it is used.

You can use #[non_exhaustive] if you want to avoid this.

Re: Can a Rust binary use incompatible versions of the same library?

#27

Post by extr »

josephg wrote: post #5
woodruffw wrote: post #2

I thought this was about loading two incompatible versions of a shared object into the same address space at first :-) The author correctly contrasts Rust (and NPM's) behavior with that of Python/pip, where only one version per package name is allowed. The Python packaging ecosystem could in theory standardize a form of package name mangling wherein multiple versions could be imported simultaneously (akin to what's c…

> (The more I work in Python, the more I think that Python's approach is actually a good one ...) I've come to the opposite conclusion. I've "git cloned" several programs in both python and ruby (which has the same behaviour) only to discover that I can't actually install the project's dependencies. The larger your gemfile / requirements.txt is, the more likely this is to happen. All it takes is a couple packages in…

I don't know, I can see it both ways. I think it depends on programming context. On one hand you're right that it's annoying and a technically unnecessary gotcha, but for some Python use cases it simplifies the mental model to simply Know which version of a particular package is running. For example, pandas and numpy are IMO bad offenders for transitive dependency issues, but it's because they're used as building blocks everywhere and are intended to be used compositionally. It's not uncommon to have to step through a data pipeline to debug it. That would become confusing if it's using 5 different major versions of pandas because each package brought it's own. Or a trip down the call stack involves multiple different versions of numpy at each level.

For web dev and something like requests, it's just not as big of a deal to have a bunch of versions installed. You don't typically use/debug that kind of functionality in a way that would cause confusion. That said, it would be definitely be great sometimes to just be like "pip, I don't care, just make it work".

Re: Can a Rust binary use incompatible versions of the same library?

#28

Post by pkolaczk »

btilly wrote: post #15

This is great for avoiding conflicts when you try to get your project running. It sucks when there is a vulnerability in a particular library, and you're trying to track all of the ways in which that vulnerable code is being pulled into your project. My preference is to force the conflict up front by saying that you can't import conflicting versions. This creates a constant stream of small problems, but avoids really…

That’s nowhere near as terrible as not being able to resolve a conflict between incompatible versions. Like half of your project can’t use Guava X but another half can’t use Guava Y, and there is no common version that works. We ran into compatibility problems with our big Java project many times and wasted months on attempting things like jar shading or classloaders. At the end of the day we use shading but that comes with its own set of annoyances like increasing the build times and allowing people to occasionally import the wrong version of library (eg. shaded instead of non-shaded). The bigger the project the more likely you’re going to hit this, and the lack of support for feature-gating dependencies in the Java ecosystem doesn’t help.

Re: Can a Rust binary use incompatible versions of the same library?

#29

Post by pkolaczk »

oefrha wrote: post #16
btilly wrote: post #15

This is great for avoiding conflicts when you try to get your project running. It sucks when there is a vulnerability in a particular library, and you're trying to track all of the ways in which that vulnerable code is being pulled into your project. My preference is to force the conflict up front by saying that you can't import conflicting versions. This creates a constant stream of small problems, but avoids really…

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

No one asked about Go here. And no, it didn’t, it’s the same PITA as in Java or maybe even worse because there are no workarounds like classloading or shading. You have no control over the transitive dependencies. The only thing you can do if there’s a conflict is asking the author to fix one of the conflicting libraries.
Post reply on HN