Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Orphaning bcachefs-tools in Debian

jonathancarter.org

Post reply on HN
Print view View original

121–130 of 224 posts

Re: Orphaning bcachefs-tools in Debian

#121

Post by chandlerswift »

cesarb wrote: post #90
agwa wrote: post #21

Earlier quoted context omitted.

The author would be fine with vendoring. The problem is that Debian doesn't allow it.

> The problem is that Debian doesn't allow it. Vendoring used to be more common, even in Debian. But then there was an important zlib vulnerability (which IIRC could be exploited through manipulated compressed data), and they had to chase and fix all the many copies of zlib embedded all over the place. To make things worse, some of them were not only old versions of zlib, but also had modified that zlib code, so each…

I'm interested in hearing more about the history behind Debian and zlib! I did some searching and the closest thing I could find was a nod to the same incident in the Upstream Guide[0]. Do you know of a place where I could read more about it?

[0]: https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_...

Re: Orphaning bcachefs-tools in Debian

#122

Post by SkiFire13 »

jonhohle wrote: post #106
yshui wrote: post #41

This is pretty dumb on Debian's part. First of all I don't understand why they insist crate dependencies must be pulled from their repository. They are just source code, not built binary. AFAIK there is no other distro that does this, what they do is that they would download crates from crates.io (`cargo vendor` is a command that does this automatically) and build against that. Arch does this, Gentoo does this, NixOS…

I’d consider the issue to be the opposite. Why does every programming language now have a package manager and all of the infrastructure around package management rather than rely on the OS package manager? As a user I have to deal with apt, ports, pkg, opkg, ipkg, yum, flatpak, snap, docker, cpan, ctan, gems, pip, go modules, cargo, npm, swift packages, etc., etc., which all have different opinions of how and where t…

Because OS packaging stuff sucks. It adds an enourmous barrier to sharing and publishing stuff.

Imagine that I make a simple OS-agnostic library in some programming language and want to publish it to allow others to use it. Do I need to package for every possible distro? That's a lot of work, and might still not cover everyone. And consider that I might not even use Linux!

A programming language will never get successful if that is what it takes to built up a community.

Moreover in the case of Rust distos are not even forced to build using crates.io. However the downside is that they have to package every single dependency version required, which due to the simplicity of publishing and updating them have become quite a lot and change much often than they would like.

The funny thing is that in the C/C++ world it's common to reimplement functionality due to the difficulty of using some dependencies for them. The result is not really different from vendoring dependencies, except for the reduced testing of those components, and this is completly acceptable to distros compared to vendoring. It makes no sense!

Re: Orphaning bcachefs-tools in Debian

#123

Post by trueismywork »

jonhohle wrote: post #106
yshui wrote: post #41

This is pretty dumb on Debian's part. First of all I don't understand why they insist crate dependencies must be pulled from their repository. They are just source code, not built binary. AFAIK there is no other distro that does this, what they do is that they would download crates from crates.io (`cargo vendor` is a command that does this automatically) and build against that. Arch does this, Gentoo does this, NixOS…

I’d consider the issue to be the opposite. Why does every programming language now have a package manager and all of the infrastructure around package management rather than rely on the OS package manager? As a user I have to deal with apt, ports, pkg, opkg, ipkg, yum, flatpak, snap, docker, cpan, ctan, gems, pip, go modules, cargo, npm, swift packages, etc., etc., which all have different opinions of how and where t…

Programming language package managers are more for development than deployment.

Re: Orphaning bcachefs-tools in Debian

#124

Post by epage »

AshamedCaptain wrote: post #17
viraptor wrote: post #5

Earlier quoted context omitted.

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really wan…

In fact, I am tired of the same issue e.g. when evaluating machine learning tools with Python. It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. Not helped by the fact that most of these projects break ABI every other day which I still think it is just plain evil. Please don't say "just use containers/virtualenvs/vendoring/whatever" because at some point you obvio…

In Rust, it is frowned upon to pin to an exact version. We do encourage people to specify what semver version range will work. We don't have great support for depending on and verifying multiple-major version ranges when a library broke compatibility but not in a way that affects you.

Re: Orphaning bcachefs-tools in Debian

#125

Post by nubinetwork »

greatgib wrote: post #76
nubinetwork wrote: post #22

Earlier quoted context omitted.

> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a blood…

I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your ca…

> For other languages like Python, there is not a need and you should not do that.

Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...

Re: Orphaning bcachefs-tools in Debian

#126

Post by rubiquity »

peppermint_gum wrote: post #30

>not even considering some hostile emails that I recently received from the upstream developer or his public rants on lkml and reddit It feels like whenever the author of bcachefs comes up, it's always because of some drama. Just the other day he clashed with Linus Torvalds: https://lore.kernel.org/lkml/CAHk-=wj1Oo9-g-yuwWuHQZU8v=VAsB... My reading is that he's very passionate, so he wants to "move fast and break thi…

Let’s not misrepresent Kent over a single incident of sending too much after a merge window. He’s extremely helpful and nice in every interaction I’ve ever read.

Re: Orphaning bcachefs-tools in Debian

#128

Post by chungy »

rubiquity wrote: post #127

Debian is going to look as ridiculous for doing this as Alma Linux is for insisting btrfs isn’t an “enterprise file system” due to it lacking RAID 5/6.

btrfs lacking RAID 5/6 is exactly the sort of thing that makes it get written off as a toy. ZFS has had working raidz[12] since 2005, raidz3 since 2007, and draid since 2021. Completely stable and enterprise-ready.

Re: Orphaning bcachefs-tools in Debian

#129

Post by SkiFire13 »

cesarb wrote: post #98
cogman10 wrote: post #18

Earlier quoted context omitted.

It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries. All rust applications are statically linked. So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.

> rust does not support dynamic linking of rust libraries. All rust applications are statically linked. That's not true. $ ldd /usr/bin/rustc linux-vdso.so.1 (0x00007fde30c44000) librustc_driver-adb68da129cf8bba.so => /lib64/librustc_driver-adb68da129cf8bba.so (0x00007fde2cc00000) libstd-aa46ce58aa106bb2.so => /lib64/libstd-aa46ce58aa106bb2.so (0x00007fde2cad3000) libc.so.6 => /lib64/libc.so.6 (0x00007fde2c8e2000) [.…

> What Rust currently doesn't support (there's a desire to fix this someday), is dynamically linking with a Rust library compiled with a different version of the Rust compiler

Note that even with the same compiler there a lot of factors that must remain the same to make the dynamic linking work properly. For example changing any compiler flag or dependency feature flag will likely result in incompatibility. The best way to guarantee that the dynamic library and the executable that loads it are compatible is to build everything as part of a single workspace (and yes, this means that if anything changes in the dynamic library crate then the executable that depend on it must also be rebuilt).

Re: Orphaning bcachefs-tools in Debian

#130

Post by koverstreet »

cesarb wrote: post #101
brnt wrote: post #96

Earlier quoted context omitted.

> What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die. They, and most others, have been doing that since their respectively beginnings, and for good reasons: everybody expects distros to fix security issues, which is greatly aided by ensuring an-as-small-as-necesary dep tree and thus a single version of…

> They, and most others, have been doing that since their respectively beginnings As I explained on another comment of mine ( https://news.ycombinator.com/item?id=41409199 ), it's not since their beginning, but since a particular zlib security incident which showed them the risks of vendored libraries. Any discussion of vendoring policy which does not consider that incident is necessarily incomplete.

That's very nearly a solved problem at this point; I get notified by bots (github, mainly) if a dependency has as a security vulnerability, and then it's very nearly a one click action to do a cargo update and commit the new lockfile.

The distro people could've been working on the tooling for automating this at the distro level (and some people in debian are doing work that would enable this); we don't need to go this insane "unbundle everything" route.

Post reply on HN