Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can a Rust binary use incompatible versions of the same library?

github.com

Post reply on HN
Print view View original

51–60 of 89 posts

Re: Can a Rust binary use incompatible versions of the same library?

#51

Post by pkolaczk »

oefrha wrote: post #34
pkolaczk wrote: post #29

Earlier quoted context omitted.

No one asked about Go here. And no, it didn’t, it’s the same PITA as in Java or maybe even worse because there are no workarounds like classloading or shading. You have no control over the transitive dependencies. The only thing you can do if there’s a conflict is asking the author to fix one of the conflicting libraries.

If someone talks about a problem I’ll damn well explain other people’s solutions as I please. And no, you resolve conflicts by not having conflicts in the first place.

> And no, you resolve conflicts by not having conflicts in the first place.

That means you can't use library A and unrelated library B together in the same project, even though you can use A alone and can use B alone. That's lack of orthogonality.

Re: Can a Rust binary use incompatible versions of the same library?

#52

Post by tialaramex »

sanxiyn wrote: post #26
whatshisface wrote: post #21

Earlier quoted context omitted.

Backwards compatibility is more difficult in Rust for many reasons. For example, you can't add a new item to an enum without creating missing-case errors everywhere it is used.

You can use #[non_exhaustive] if you want to avoid this.

Right. This even tidily forbids the exhaustive matching for other people's code (ie requires them to write a default) using your crate but still allows it within the crate, reasoning that you should know which values you added even if you never promise an exhaustive list to your users.

Re: Can a Rust binary use incompatible versions of the same library?

#53

Post by stouset »

gmueckl wrote: post #20
ComputerGuru wrote: post #8

Earlier quoted context omitted.

I’m confused. If you rely on a feature introduced in X.y.z why would you specify X.y.0 to begin with (and not just X.y.z)? In practice, usual rust projects that have not put a ton of work into their dependencies encode X.y.z in Cargo.toml matching the current release at the time they developed the system. So you get at worst an unnecessarily higher version requirement but never a lower one. Moreover, rust semver woul…

It's easy to accidentally ship a minimum version requirement that is out of date when you also consistently use lock files pinned to newer versions. The code may silently depend on something introduced in a newer version pulled in by the lock file.

Just have CI build with the minimum and the maximum.

Re: Can a Rust binary use incompatible versions of the same library?

#54

Post by dathinab »

gorgoiler wrote: post #10

For fun, you could add this to Python and I think it would it cover a lot of edge cases? You would need: A function v_tree_install(spec) which installs a versioned pypi package like “foo=3.2” and all its dependencies in its own tree, rather than in site-packages. Another pair of functions v_import and v_from_import to wrap importlib with a name, version, and symbols. These functions know how to find the versioned pac…

I have though about this a bunch (and have been annoyed by it a bunch).

But the main issue here is somewhat designed around a "scripts and folder of scripts from a package" design principle while such a loading system would fundamentally need to always work in terms of packages. E.g. you wouldn't execute `main.py` but `package:main`. (Through this is already the direction a bunch of tooling moved to, e.g. poetry scripts, some of the WSGI and especially more modern ASGI implementations etc.)

Another issue is that rust can reliable detect type collisions of the same type of two different versions and force you to fix them.

With a lot of struct type annotations on python and tooling like mypy this might be possible (with many limitations) but as of today it in practice likely will not be caught. Sometimes that is what you want (ducktyping happens to work). But for any of the reflection/inspection heavy python library this is a recipe for quite obscure errors somewhere in not so obvious inspection/auto generation/metaclass related magic code. Python can't, escept it can

Anyway technically it's possible, you can put a version into __qualname__, and mess with the import system enough to allow imports to be contextual based on the manifest of the module they come from. (Through you probably would not be fully standard conform python, but we are speaking about dynamic patching pythons import system, there is nothing standard about it)

Re: Can a Rust binary use incompatible versions of the same library?

#56

Post by oefrha »

pkolaczk wrote: post #51
oefrha wrote: post #34

Earlier quoted context omitted.

If someone talks about a problem I’ll damn well explain other people’s solutions as I please. And no, you resolve conflicts by not having conflicts in the first place.

> And no, you resolve conflicts by not having conflicts in the first place. That means you can't use library A and unrelated library B together in the same project, even though you can use A alone and can use B alone. That's lack of orthogonality.

No, you seriously discourage libraries from breaking compatibility by removing the possibility to hide behind different pinned versions and version ranges.

Re: Can a Rust binary use incompatible versions of the same library?

#57

Post by oefrha »

MrJohz wrote: post #49
oefrha wrote: post #33

Earlier quoted context omitted.

It’s c and c/v2, not c@1.0.0, c@1.0.5, c@1.0.10, c@1.1.3, c@2.0.0, c@2.3.1, ... Each necessary because packages in the middle have decided to pin versions or add upper bounds to work around bugs. That’s a huge difference.

FWIW, I've just pulled up a pretty large project I work on using NPM, and almost all of the duplicate dependencies had different major versions. Most of the ones that had the same major version were 0.x dependencies with different minor versions. So I'm still not convinced that Go's approach is materially different here - certainly in terms of the practical output, NPM does a good job of ensuring that the fewest numb…

You forgot the part where npm people release new major versions for very little reason all the time, because there’s nothing stopping them. Go authors on the other hand are generally really reluctant to change to a new path. Go to a relatively large go codebase and count the v2s. Then do v3.

Coming back to a midsized JavaScript codebase after a few months and trying to upgrade to new major versions of things have always been a shitshow.

Re: Can a Rust binary use incompatible versions of the same library?

#58

Post by virtualritz »

josephg wrote: post #5
woodruffw wrote: post #2

I thought this was about loading two incompatible versions of a shared object into the same address space at first :-) The author correctly contrasts Rust (and NPM's) behavior with that of Python/pip, where only one version per package name is allowed. The Python packaging ecosystem could in theory standardize a form of package name mangling wherein multiple versions could be imported simultaneously (akin to what's c…

> (The more I work in Python, the more I think that Python's approach is actually a good one ...) I've come to the opposite conclusion. I've "git cloned" several programs in both python and ruby (which has the same behaviour) only to discover that I can't actually install the project's dependencies. The larger your gemfile / requirements.txt is, the more likely this is to happen. All it takes is a couple packages in…

> Its a completely unnecessary landmine. [...]

This! I have transitive conflicts almost every time I clone an exsting Python repo. It's one of the main reasons why Python == PITA^3 in my head.

Maybe it's the kind of Python repos I clone. Mostly ML/diffusion & computer graphics stuff.

As a Rustacean I had hoped that the Rhye + uv combo would 'fix' this but I now understand they won't.

Re: Can a Rust binary use incompatible versions of the same library?

#59

Post by seritools »

stouset wrote: post #3
woodruffw wrote: post #2

I thought this was about loading two incompatible versions of a shared object into the same address space at first :-) The author correctly contrasts Rust (and NPM's) behavior with that of Python/pip, where only one version per package name is allowed. The Python packaging ecosystem could in theory standardize a form of package name mangling wherein multiple versions could be imported simultaneously (akin to what's c…

> dependency graph spaghetti The worst spaghetti comes from hard dependencies on minor versions and revisions. I will die on the hill that you should only ever specify dependencies on “at least this major-minor (and optionally and rarely revision for a bugfix)” in whatever the syntax is for your preferred language. Excepting of course a known incompatibility with a specific version or range of versions, and/or develo…

> In Rust, Cargo makes this super easy: “x.y.z” means “>= x.y.z, With the added special case of `0.x.y` meaning `>= 0.x.y, < 0.(x+1).0`, going beyond what semver specifies.

Re: Can a Rust binary use incompatible versions of the same library?

#60

Post by skitter »

whatshisface wrote: post #21
oefrha wrote: post #16

Earlier quoted context omitted.

Go got this right: you want an incompatible version, you have to use a different import path. Then you can only pick one version (which is deterministically the lowest possible version) for a certain import path, not a hundred different versions. Also forces people to actually take backwards compatibility seriously.

Backwards compatibility is more difficult in Rust for many reasons. For example, you can't add a new item to an enum without creating missing-case errors everywhere it is used.

Adding an enum variant can be backwards incompatible just as easily in languages that don't do this, you just don't get to see the error at compile time.
Post reply on HN