Orphaning bcachefs-tools in Debian • Hacker News Board

Re: Orphaning bcachefs-tools in Debian

#11

Post by 1oooqooq » Sat, Aug 31, 2024, 12:28 PM UTC

viraptor wrote: post #5

JonChesterfield wrote: post #3

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set…

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

it is rust specific because rust is the first attemp to replace proper system engineering languages with one that while nicer on memory management and overall ergonomics, is worse in tooking. cargo brings many malpractices from java/JavaScript (maven, npm, etc) that were always shunned in systems engineering and, mark my words, will be a security nightmare for linux in the near future.

Re: Orphaning bcachefs-tools in Debian

#12

Post by viraptor » Sat, Aug 31, 2024, 12:30 PM UTC

JackSlateur wrote: post #8

viraptor wrote: post #5

Earlier quoted context omitted.

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

if you deal with filesystems, you really want strict versions How are doing everybody else ? What is specific with filesystems, here ? This is a bad dependency management, nothing more. Vendoring is a PITA.

Level of potentially messing up. If you have a chat app, relax the deps and it crashes - oh well. But if you have a tool which can cause data loss, you really want to be sure everyone's running what you tested.

It's similar to why restic (backup software) has official, static, reproducible binaries in their releases.

Re: Orphaning bcachefs-tools in Debian

#13

Post by JonChesterfield » Sat, Aug 31, 2024, 12:30 PM UTC

JackSlateur wrote: post #8

viraptor wrote: post #5

Earlier quoted context omitted.

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

if you deal with filesystems, you really want strict versions How are doing everybody else ? What is specific with filesystems, here ? This is a bad dependency management, nothing more. Vendoring is a PITA.

Filesystems are complicated and have nasty failure modes. Strict versioning reduces the impact of software development QoI properties, i.e. that basically all code is substantially wrong.

Re: Orphaning bcachefs-tools in Debian

#14

Post by rlpb » Sat, Aug 31, 2024, 12:33 PM UTC

JonChesterfield wrote: post #3

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set…

Maintenance is much more practical if everything in your dependency tree uses the same version of every dependency. Achieving this in practice is much of the work distribution maintainers have to do.

Doing this has a couple of key advantages:

1. when a security vulnerability needs patching in a stable distribution release, the security team only have to patch one version instead of a dozen or more different versions bundled all over the place

2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version

In an ecosystem where it's considered acceptable to "pin" versions of dependencies and also call any system that doesn't use the pinned versions "unsupported", both of the above two cases become impractical.

Whether you use shared libraries or static libraries, the above matters still exist.

Re: Orphaning bcachefs-tools in Debian

#15

Post by 1oooqooq » Sat, Aug 31, 2024, 12:34 PM UTC

mananaysiempre wrote: post #7

See also on the same general topic: “The modern packager’s security nightmare” (2021) [1] by Gentoo maintainer Michał Górny, and Drew DeVault’s posts on why FOSS developers should not distribute software directly or try to impose their will on distros[2,3]. [1] https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packag... , discussed at https://news.ycombinator.com/item?id=26203853 (320 points, 278 comments) [2] ht…

exactly. the sad part is that this is being coopted into an aegument for flatpack! it's insane the discussions on some distros.

they cannot see that by giving in to static libs you make every little application maintainer into a full-time distro maintainer responsible for it forever. ... good luck with that.

Re: Orphaning bcachefs-tools in Debian

#16

Post by mananaysiempre » Sat, Aug 31, 2024, 12:34 PM UTC

h2odragon wrote: post #9

this is "static vs shared libraries" fight again. no one remembers the first iterations, and there's more layers of "stuff" in the way now, so now one sees the actual issue for what it is. "shared libs" was the product of storage constraints; a bobble in the smooth pace of progress that shouldn't be needed now. Our faith in "the way things are done now is the right way" and our (justified) fear of messing with the fo…

It’s rather the “vendoring is hostile to distros” fight. It’s adjacent to the shared vs static one but not the same: it’s absolutely possible to have statically linked binaries tracking distro-wide dependency versions, provided the distro’s build system is robust and automated enough. Not all are.

Re: Orphaning bcachefs-tools in Debian

#17

Post by AshamedCaptain » Sat, Aug 31, 2024, 12:34 PM UTC

viraptor wrote: post #5

JonChesterfield wrote: post #3

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set…

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

In fact, I am tired of the same issue e.g. when evaluating machine learning tools with Python. It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. Not helped by the fact that most of these projects break ABI every other day which I still think it is just plain evil.

Please don't say "just use containers/virtualenvs/vendoring/whatever" because at some point you obviously want these "containers" to use your real hardware. And your GPU driver is only going to be at one version, no matter how many statically linked executables you have.

Re: Orphaning bcachefs-tools in Debian

#18

Post by cogman10 » Sat, Aug 31, 2024, 12:35 PM UTC

viraptor wrote: post #5

JonChesterfield wrote: post #3

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set…

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries. All rust applications are statically linked.

So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.

Re: Orphaning bcachefs-tools in Debian

#19

Post by viraptor » Sat, Aug 31, 2024, 12:36 PM UTC

h2odragon wrote: post #9

this is "static vs shared libraries" fight again. no one remembers the first iterations, and there's more layers of "stuff" in the way now, so now one sees the actual issue for what it is. "shared libs" was the product of storage constraints; a bobble in the smooth pace of progress that shouldn't be needed now. Our faith in "the way things are done now is the right way" and our (justified) fear of messing with the fo…

This is unrelated to shared libraries. Those strict versions are for compile-time dependencies.

Re: Orphaning bcachefs-tools in Debian

#20

Post by aragilar » Sat, Aug 31, 2024, 12:36 PM UTC

haileys wrote: post #10

JonChesterfield wrote: post #3

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set…

I think this is the same shape of issue that I’ve experienced with Debian for as long as I’ve used it - close to 15 years now. Debian is a great OS, but it targets stability and long term support for its releases. That just isn’t compatible with newer, faster moving software that’s still working towards stability. I remember it being an issue when I was playing around with Mono around 2010 ish, and it’s an issue now…

What I think is somewhat interesting in this case (if I've read the post correctly) half the changes were for newer versions of dependencies. If that's the case, then I'm inclined to start wondering what's going on with bcachefs upstream (and the whole dependency tree), if they're not keeping up with versions.