Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The C-Shaped Hole in Package Management

nesbitt.io

Post reply on HN
Print view View original

61–70 of 77 posts

Re: The C-Shaped Hole in Package Management

#61

Post by dminik »

rwmj wrote: post #2

Please don't. C packaging in distros is working fine and doesn't need to turn into crap like the other language-specific package managers. If you don't know how to use pkgconf then that's your problem.

No, just no.

Using system/distro packages is great when you're writing server software and need your base system to be stable.

But, for software distributed to users, this model fails hard. You generally need to ship across OSs, OS versions and for that you need consistent library versions. Your software being broken because a distro maintainer has decided that a 3 year old version of your dependency is close enough is terrible.

Re: The C-Shaped Hole in Package Management

#62

Post by MiiMe19 »

dminik wrote: post #61
rwmj wrote: post #2

Please don't. C packaging in distros is working fine and doesn't need to turn into crap like the other language-specific package managers. If you don't know how to use pkgconf then that's your problem.

No, just no. Using system/distro packages is great when you're writing server software and need your base system to be stable. But, for software distributed to users, this model fails hard. You generally need to ship across OSs, OS versions and for that you need consistent library versions. Your software being broken because a distro maintainer has decided that a 3 year old version of your dependency is close enough…

If you software is not being distributed by that distribution and is using some external download tool, it is inherently not supported and the only way to make sure it works is to compile from source.

Re: The C-Shaped Hole in Package Management

#63

Post by aa-jv »

bluGill wrote: post #26
aa-jv wrote: post #3

Earlier quoted context omitted.

^ This. Plus, we already have great C package management. Its called CMake.

CMake is not a package management tool, it is a build tool. It can be abused to do package management, but that isn't what it is for.

It's a perfectly cromulant package manager.

Re: The C-Shaped Hole in Package Management

#64

Post by Joker_vD »

rwmj wrote: post #45
Joker_vD wrote: post #28

Earlier quoted context omitted.

Well, if you're fine with using 3-year old versions of those libraries packaged by severely overworked maintainers who at one point seriously considered blindly converting everything into Flatpaks and shipping those simply because they can't muster enough of manpower, sure. "But you can use 3rd party repositories!" Yeah, and I also can just download the library from its author's site. I mean, if I trust them enough t…

If this is a concern (which it rarely is) then you can pitch in with distro packaging. Volunteers are always welcome. > "But you can use 3rd party repositories!" That's not something I said.

> That's not something I said.

This was a pre-emptive rebuttal to the non-answer "well, you're not limited to the official repositories, so apt/yum/etc. is absolutely fine, use only it" I always get unless I include this very rebuttal.

> then you can pitch in with distro packaging. Volunteers are always welcome.

Or I can just do something more useful and less straining?

Re: The C-Shaped Hole in Package Management

#65

Post by dminik »

MiiMe19 wrote: post #62
dminik wrote: post #61

Earlier quoted context omitted.

No, just no. Using system/distro packages is great when you're writing server software and need your base system to be stable. But, for software distributed to users, this model fails hard. You generally need to ship across OSs, OS versions and for that you need consistent library versions. Your software being broken because a distro maintainer has decided that a 3 year old version of your dependency is close enough…

If you software is not being distributed by that distribution and is using some external download tool, it is inherently not supported and the only way to make sure it works is to compile from source.

If you compile from source, but your distro is shipping library version that is incompatible with the app, you're still screwed.

This is why flatpaks/snaps/app images have been taking off. Devs don't have time for bugs caused by incompatible libraries. Distro packagers don't have time to properly test the thousands of packages they have to change to satisfy their 1 shared library version policy.

Re: The C-Shaped Hole in Package Management

#66

Post by em-bee »

duped wrote: post #54
amiga386 wrote: post #49

Earlier quoted context omitted.

It's hardly unrealistic - most free software has been packaged, by each distro. Very handy for the developer: just email the distro maintainers (or post on your mailing list) that the new version is out, they'll get round to packaging it. Very handy for the user, they just "apt install foo" and ta-da, Foo is installed. That was very much the point of using a Linux distro (the clue is in the name!) Trying to work in a…

User now waits for 3rd party "maintainers" to get around to manipulating the software they just want to use from the 1st party developer they have a relationship with. If ever. I understand this is how distros work. What I'm saying is that the distros are wrong, this is a bad design. It leads to actual bugs and crashes for users. There have been significant security mistakes made by distro maintainers. Distros strip…

distributions started out with solving the problem that most developers at that time didn't even bother to build ready to run packages. they couldn't, because there were to many different architectures that not everyone had access to. so developers had to rely on users to build the applications for themselves. distributions then organized around that to make this easier for users. that's how the port system in BSD came about. linux distributions went a step further and built distributable binaries.

the problem was to not predict that developers would want more control over the build of their applications, which, thanks to architectures consolidating, became easier because now a single binary will reach the majority of your userbase. and the need to support multiple versions of the same library or app in the package manager. that support should have been there from the start, and now its difficult to fix that.

so it's unfair to say distros are wrong. yes, it's not an ideal design, but this is more of an accident of history, some lack of foresight, and the desire to keep things simple by having only the newest version of each package.

there is a conflict between the complexity of supporting multiple package versions vs the complexity of getting applications to work with the specific library versions the distro supports. when distros started out it looked like the latter would be better for everyone. distributions tended to have the latest versions of libraries and fixing apps to work with those benefited the apps in most cases.

Re: The C-Shaped Hole in Package Management

#67

Post by imtringued »

pornel wrote: post #53

Has anyone here even read the article?! All the comments here assume they're building a package manager for C! They're writing a tool to discover and index all indirect dependencies across languages, including C libraries that were smuggled inside other packages and weren't properly declared as a dependency anywhere. "Please don't" what? Please don't discover the duplicate and potentially vulnerable C libraries that…

Yeah it's pretty weird how people assume that -l is supposed to work in gcc/clang across distributions, but somehow deriving which OS package gives you that lib.so file is the devil.

Re: The C-Shaped Hole in Package Management

#68

Post by etbebl »

I get that the scope of the article is a bit larger than this, but it's a pet peeve of mine when authors acknowledge the advantages of conda and then dismiss it for...silly? reasons. It kind of sounds like they just don't know many people using it, so they assume something must be wrong with it.

> If you don’t need compiled extensions, Conda is more than you need.

Am I missing something or isn't that exactly the problem we're talking about here?

> And even when you do need it, conda environments are heavier than virtual environments and the resolver used to be infamously slow. Mamba exists largely because conda’s dependency resolution took forever on nontrivial environments.

Like it says here, speed isn't a problem anymore - mamba is fast. And it's true that the environments get large; maybe there's bloat, but it definitely does share package versions across environments when possible, while keeping updates and such isolated to the current environment. Maybe there's a space for a language package manager that tries to be more like a system package manager by updating multiple envs at once while staying within version constraints to minimize duplication, but idk if many developers would think that is worth the risk.

Re: The C-Shaped Hole in Package Management

#69

Post by rurban »

This comes up every ten years or so, and is a solved problem. Any decent distro has tools to scan the dependencies of each binary via ldd, to check if its deps are correct.

His example numpy shipping its own libblas.so, has the speciality that it's runtime loaded, so ldd will not find it, but the runtime dep is in the MANIFEST. And seeing that is not in a standard path concludes that is a private copy, that needs to be updated seperately if broken.

No other hole than in his thinking and worrying.

Re: The C-Shaped Hole in Package Management

#70

Post by elehack »

etbebl wrote: post #68

I get that the scope of the article is a bit larger than this, but it's a pet peeve of mine when authors acknowledge the advantages of conda and then dismiss it for...silly? reasons. It kind of sounds like they just don't know many people using it, so they assume something must be wrong with it. > If you don’t need compiled extensions, Conda is more than you need. Am I missing something or isn't that exactly the prob…

Mamba is fast, and Pixi is also fast + sands a lot of the rough edges off the Conda experience (with project/environment binding and native lock files).

Not perfect, but pretty good when uv isn't enough for a project or deployment scenario.

Post reply on HN