Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Orphaning bcachefs-tools in Debian

jonathancarter.org

Post reply on HN
Print view View original

211–220 of 224 posts

Re: Orphaning bcachefs-tools in Debian

#211

Post by rcxdude »

greatgib wrote: post #76
nubinetwork wrote: post #22

Earlier quoted context omitted.

> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a blood…

I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your ca…

No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime.

It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything).

Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.

Re: Orphaning bcachefs-tools in Debian

#212

Post by ris »

AshamedCaptain wrote: post #173
ris wrote: post #167

Earlier quoted context omitted.

> It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. This is what I call "version soup". The idea that every project can choose an near-arbitrary set of very exact version numbers of dependencies and expect it to work. And that every project on earth has the burden of continually stirring their soup, often through a tool like dependabot, hoping (cross fingers) no c…

> In nixpkgs we attempt to address this situation for the python ecosystem by providing one version of each package (with few exceptions) per release. But in return, we put in work to make sure those versions actually work with each other - generally by getting the projects' test suites integrated into the build system. The idea is that an app built to depend on nixpkgs packages should be able to expect to do depende…

Essentially no, but

- most Linux distributions don't reach very far into e.g. the python ecosystem (python packages that exist are generally there to support packaged applications)

- nixpkgs isn't linux specific - most packages work on macos, and nixpkgs happily works on top of other distributions.

Re: Orphaning bcachefs-tools in Debian

#213

Post by rcxdude »

aragilar wrote: post #20
haileys wrote: post #10

Earlier quoted context omitted.

I think this is the same shape of issue that I’ve experienced with Debian for as long as I’ve used it - close to 15 years now. Debian is a great OS, but it targets stability and long term support for its releases. That just isn’t compatible with newer, faster moving software that’s still working towards stability. I remember it being an issue when I was playing around with Mono around 2010 ish, and it’s an issue now…

What I think is somewhat interesting in this case (if I've read the post correctly) half the changes were for newer versions of dependencies. If that's the case, then I'm inclined to start wondering what's going on with bcachefs upstream (and the whole dependency tree), if they're not keeping up with versions.

It looks like the two packages that were bumped forward are ones where the changes were very minimal (it's not obvious why errno even bumped their 'major' version, and udev seem to do it on every release even when backwards compatible). Also errno 0.4 doesn't seem to exist, so I think that's a typo.

(There is a somewhat frustrating trend in rust packages for them to sit on 0.x versions for a very long time, even if they're fairly heavily used in the ecosystem and pretty stable in practice)

From what the author of bcachefs is saying, the forward moves were not a problem, it was moving bindgen backwards which broke the build, meaning the packaged version of bcachefs-tools stayed very out of date, causing him to get bug reports for known and fixed fairly serious bugs (ones which rendered machines unbootable).

Re: Orphaning bcachefs-tools in Debian

#214

Post by brnt »

IshKebab wrote: post #207
brnt wrote: post #96

Earlier quoted context omitted.

> What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die. They, and most others, have been doing that since their respectively beginnings, and for good reasons: everybody expects distros to fix security issues, which is greatly aided by ensuring an-as-small-as-necesary dep tree and thus a single version of…

> everybody expects distros to fix security issues IMO this is a huge mistake in the Linux distro world. It doesn't scale. It's similar to the weird idea that Debian should contain every program in the world and if the authors want their programs to be installable on Debian they must make a Debian package for it. And Fedora and Arch and Gentoo and... Clearly bonkers.

Debian does not prevent you from installing stuff from elsewhere, it's just that their packages come with certain availability and stability guarantees. You may not care for them, but clearly bonkers it is clearly not.

Re: Orphaning bcachefs-tools in Debian

#215

Post by brnt »

koverstreet wrote: post #130
cesarb wrote: post #101

Earlier quoted context omitted.

> They, and most others, have been doing that since their respectively beginnings As I explained on another comment of mine ( https://news.ycombinator.com/item?id=41409199 ), it's not since their beginning, but since a particular zlib security incident which showed them the risks of vendored libraries. Any discussion of vendoring policy which does not consider that incident is necessarily incomplete.

That's very nearly a solved problem at this point; I get notified by bots (github, mainly) if a dependency has as a security vulnerability, and then it's very nearly a one click action to do a cargo update and commit the new lockfile. The distro people could've been working on the tooling for automating this at the distro level (and some people in debian are doing work that would enable this); we don't need to go thi…

Before broadband, and before forges like github with free CI and bots, which is all fairly recent, distros packaged everything was a godsend, not insane.

Re: Orphaning bcachefs-tools in Debian

#216

Post by nequo »

jmb99 wrote: post #204
CJefferson wrote: post #35

Earlier quoted context omitted.

Where are those things in the standard library of C or C++, which most software is written in? Most C Debian packages have a few dependencies in my experience, they are just “standard”, because they are old and stable.

Parsing command line arguments, reading/manipulating binary data, memset_s are of course all available in the C standard library. I’m not familiar with rust, but if it’s true that dependencies are required for those types of basic operations... yikes, to say the least. If you include boost (which, admittedly, is a heavy dependency, but still a single one), everything mentioned is possible.

> If you include boost (which, admittedly, is a heavy dependency, but still a single one)

Then in principle, we could curate a set of useful Rust crates, treat the set as a single dependency, and be on the same page as C++ with Boost.

Re: Orphaning bcachefs-tools in Debian

#217

Post by zamadatix »

kijin wrote: post #209
zamadatix wrote: post #160

Earlier quoted context omitted.

I didn't get the impression anyone bothered the maintainers to do this, the article implies they chose to package it on their own accord.

Yes, and they chose to stop packaging it, again on their own accord. Except this time, they seem to have made the decision because upstream was bothering them too much.

No arguments there, I'm more talking what that means for the "build and distribute your own dev/rpm" part that follows. Why are the only options "do work for maintainers" or "provide a prebuilt package for the distro", what happened to "nobody said this needed to be done yet"?

Re: Orphaning bcachefs-tools in Debian

#218

Post by aseipp »

rlpb wrote: post #181
aseipp wrote: post #108

Earlier quoted context omitted.

Not all distributions suffer from this, because they have the tooling to patch all crate versions given a name. Compile times are actually the worst part but can be mitigated. But these distros also fundamentally accept the language's compilation model in question, so they have to develop solutions if they want to supply software to their users rather than make up excuses. Most distros don't like vendoring, for reaso…

Tooling to apply the patches is trivial. This isn't the problem I'm describing. As the sibling comment says, the issue is when the patch doesn't apply cleanly to some subset of versions in use, or even worse if they appear to apply cleanly but leave behind logic bugs. If distributions did approach this by "distro-patching" specific versions, then that would reintroduce the upstream complaint that distros are using ve…

Yeah, I don't think either of these are really big problems. People identify the vuln, the fix, and they tend to share the patches for backports across versions + across distros, and/or immediately update downstream users to fixed versions because, again, the nifty lockfiles literally tell you if they are vulnerable as they contain the dep graph. You query every package's dep graph and look at the bounds. Maintainers with RUSTSEC advisories will yank the crate on crates.io, forcing upgrades across the stack. You can apply patches to most versions of a crate (again: you can find all of them!) and loosen or make more targeted fixes if needed. But even if 30 versions are vulnerable, in practice 30 patches don't need to be written in the p99 case. So I just disagree with the sibling on that, based on my experience. Most of these packages aren't the velocity of the Linux kernel. They aren't rewriting 10,000 lines of code a version so you have to re-interrogate the fix.

But most importantly, I don't see any argument this is all meaningfully more work than all the manual shit Debian maintainers do to get all this stuff working, on top of the fact they try to ship frankenstein versions of packages that silently break and piss off users (every distro ships some busted packages but it's a matter of give and take.) Other maintainers of other distros don't spend time on all the stuff the OP spent time on. They don't spend time loosening lockfile bounds on Rust crates for filesystem-sensitive tooling(!!!) in an effort to make a 35-year-old cultural rule about C/C++'s compilation model apply to them. Mostly, they run some script like 'import crates.io package XYZ version 1.2.3', commit the result, and get it code reviewed and merged. They can then spend time on other things. And often, these same processes are used for all updates, including many security updates. This stuff is insanely well oiled in other projects.

Sorry to be the hater, but this is squarely a Debian dysfunction as far as I can see? If they spent more time actually solving infrastructural problems they had, they would be able to spend time on actually delivering more working software to their users in the world they actually exist in.

Re: Orphaning bcachefs-tools in Debian

#219

Post by Brian_K_White »

Brian_K_White wrote: post #56

Sure buddy. Not only Debian but the kernel itself too. Those immature unproven little projects and their silly "best practices". What do Debian or the kernel know about building and maintaining bedrock software? https://www.phoronix.com/news/Linus-Torvalds-Bcachefs-Regret...

Apologies I got this kinda backwards. The Debian article was written from Debians perspective not bcachefs. "sure buddy" and the rest of the sarcasm was directed at bcachefs.

Re: Orphaning bcachefs-tools in Debian

#220

Post by Arch-TK »

ralferoo wrote: post #198
Arch-TK wrote: post #196

Earlier quoted context omitted.

> The repo is literally called "linux-stable-rc" It's not? What repo? The only two repos which are involved are https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... which is mainline (implicitly) which is where these patches would land and git://evilpiepirate.org/bcachefs.git which is the source repo for the PR. The only branches being referenced are "master" (implicitly) for mainline and the tag "bcachefs…

> It's not? What repo? https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux...

Yes I can imagine there are a bunch of repos with "stable" in their name in git.kernel.org but where did you find a reference to this repo in that thread?

There are repos on git.kernel.org for microemacs, that doesn't mean this thread relates to microemacs.

Post reply on HN