Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can a Rust binary use incompatible versions of the same library?

github.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: Can a Rust binary use incompatible versions of the same library?

#41

Post by ahupp »

gorgoiler wrote: post #10

For fun, you could add this to Python and I think it would it cover a lot of edge cases? You would need: A function v_tree_install(spec) which installs a versioned pypi package like “foo=3.2” and all its dependencies in its own tree, rather than in site-packages. Another pair of functions v_import and v_from_import to wrap importlib with a name, version, and symbols. These functions know how to find the versioned pac…

You might be able to do this transparently with a [MetaPathFinder](https://docs.python.org/3/library/importlib.html#importlib.a...), the only trickyness would be replacing the lookup in sys.modules which I don't think has has an official interface.

Re: Can a Rust binary use incompatible versions of the same library?

#42

Post by amelius »

anonymoushn wrote: post #39
amelius wrote: post #35

Earlier quoted context omitted.

How would that work for malloc? How can you have two different functions manage the same heap?

"The same heap" isn't a coherent concept here. Your malloc-implementing memory allocator has some global state, and that state has some pointers to some addresses it got from mmap and some metadata about how long those spans of memory are, which parts are unused, and how long the values it has returned from malloc previously are. If you managed to use two of these, they would each contain data referring to different…

But malloc calls sbrk(), right?

Re: Can a Rust binary use incompatible versions of the same library?

#44

Post by dwattttt »

amelius wrote: post #42
anonymoushn wrote: post #39

Earlier quoted context omitted.

"The same heap" isn't a coherent concept here. Your malloc-implementing memory allocator has some global state, and that state has some pointers to some addresses it got from mmap and some metadata about how long those spans of memory are, which parts are unused, and how long the values it has returned from malloc previously are. If you managed to use two of these, they would each contain data referring to different…

But malloc calls sbrk(), right?

I can't speak to multiple users of sbrk, I assume that would fail. That's a property of sbrk though, not of malloc (or memory allocation in general); on Windows, you can have as many implementors of malloc as you want, so long as malloc/free calls happen to the same implementor. Each malloc implementor just asks for anonymous backing pages for their own heaps (via VirtualAlloc/mmap).

Re: Can a Rust binary use incompatible versions of the same library?

#45

Post by alkonaut »

How does this work? Assume that the log crate in its internal state has a lock it uses for synchronizing writing to some log endpoint. If I have two versions of log in my process then they must have two copies of their internal state. So they both point to the same log endpoint, but they have one mutex each? That means it "works" at compile time but fails at runtime? That's the worst kind of "works!"

Or if I depend transitively on two versions of a library (e.g. a matrix math lib) through A and B and try to read a value from A and send it into B. Then presumably due to type namespacing that will fail at compile time?

So the options when using incompatible dependencies are a) it compiles, but fails at runtime, b) it doesn't compile, or c) it compiles and works at runtime?

Re: Can a Rust binary use incompatible versions of the same library?

#46

Post by gary_0 »

WD-42 wrote: post #32

This isn’t a magic bullet. Using multiple versions of the same crate can still blow up your project. For example, the compiler error in this example: note: perhaps two different versions of crate `smithay_client_toolkit` are being used? https://github.com/pop-os/launcher/issues/237

Ah, I was wondering what would happen if you're using a type from lib-v2 and an intermediary library passes you that type from lib-v1, and the type has changed internally. Good to know the Rust compiler is set up to catch that.

(I've seen cases where that happens with C and C++ software, and things seem to compile and run... until everything explodes. Fun times.)

Re: Can a Rust binary use incompatible versions of the same library?

#47

Post by unusual-name »

amelius wrote: post #42
anonymoushn wrote: post #39

Earlier quoted context omitted.

"The same heap" isn't a coherent concept here. Your malloc-implementing memory allocator has some global state, and that state has some pointers to some addresses it got from mmap and some metadata about how long those spans of memory are, which parts are unused, and how long the values it has returned from malloc previously are. If you managed to use two of these, they would each contain data referring to different…

But malloc calls sbrk(), right?

It depends on the implementation. For example, jemalloc can use sbrk(), sbrk() and mmap() or just mmap(), depending on what your os supports.

Re: Can a Rust binary use incompatible versions of the same library?

#49

Post by MrJohz »

oefrha wrote: post #33
MrJohz wrote: post #24

Earlier quoted context omitted.

But surely you can still run into the same issue: I import a@1 and b@1 a@1 transitively depends on c@1 b@1 transitively depends on c@2 Even with different import paths, I still have two different versions of c in my codebase. It'll just be that one of them is imported as "c" and the other will be imported as "c/v2" - but you don't need to worry about that, because that's happening in transitive dependencies that you'…

It’s c and c/v2, not c@1.0.0, c@1.0.5, c@1.0.10, c@1.1.3, c@2.0.0, c@2.3.1, ... Each necessary because packages in the middle have decided to pin versions or add upper bounds to work around bugs. That’s a huge difference.

FWIW, I've just pulled up a pretty large project I work on using NPM, and almost all of the duplicate dependencies had different major versions. Most of the ones that had the same major version were 0.x dependencies with different minor versions.

So I'm still not convinced that Go's approach is materially different here - certainly in terms of the practical output, NPM does a good job of ensuring that the fewest number of different versions will get installed for each dependency.

Re: Can a Rust binary use incompatible versions of the same library?

#50

Post by yorwba »

alkonaut wrote: post #45

How does this work? Assume that the log crate in its internal state has a lock it uses for synchronizing writing to some log endpoint. If I have two versions of log in my process then they must have two copies of their internal state. So they both point to the same log endpoint, but they have one mutex each ? That means it "works" at compile time but fails at runtime? That's the worst kind of "works!" Or if I depend…

If the log endpoint is external to your process and two different copies of the logging crate in the same process writing to it cause problems, two identical copies of the logging crate in different processes will likely also cause problems. The solution here is global synchronzation, not just within one process.

If the log endpoint is internal to your process, how did you end up with two independent mutexes guarding (or not guarding) access to the same resource? It should be wrapped in a shared mutex as soon as you create it, and before passing it to the different versions of the logging crate. And unless you use unsafe, Rust's ownership model forces you to do that, because it forbids having two overlapping mutable references at the same time.

Post reply on HN