Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The modern packager’s security nightmare

blogs.gentoo.org

Post reply on HN
Print view View original

211–220 of 282 posts

Re: The modern packager’s security nightmare

#211

Post by orf »

syllogism wrote: post #181
orf wrote: post #178

Earlier quoted context omitted.

You’re completely wrong and this advice is somewhat harmful. What you’re describing is how a Python application should be managed. Not a library. Libraries should absolutely not lock their advertised dependencies to arbitrary point-in-time versions for fairly obvious reasons. Picking a suitable dependency specifier depends heavily on the maturity of the library you’re using and if you need any specific features added…

Everyone needs commands like "pip install 'spacy==2.3.5'" to work reliably in the future, so that you can go back and bisect errors. You need to be able to get back to a particular known-good state, and work through changes I'm not saying we pin our dependencies to exact specific versions, but we absolutely do set an upper bound, usually to the minor version.

You should use a tool that supports a lockfile, not pip directly. I recommend Poetry.

Re: The modern packager’s security nightmare

#212

Post by drran »

lifthrasiir wrote: post #171
bscphil wrote: post #86

Earlier quoted context omitted.

> They should be cut out of the loop. I disagree. This is how you get the Google Play store or the "freeware" app marketplace. It sucks. As a user, I'm quite happy to continue using a traditional distribution even if it means I don't get to use a handful of flashy programs by developers that disagree with the concept of maintainers. So far that choice has been much more positive than negative for me, and I'm doing my…

> As a user, I'm quite happy to continue using a traditional distribution even if it means I don't get to use a handful of flashy programs by developers that disagree with the concept of maintainers. As a user, I'm afraid that you don't seem to be a representative of typical users. I would be happy to use a traditional distribution if they don't break on updates, which is still not the case. It's clear that tradition…

Distributions are not breaking software, they are just distributing broken software. Blame upstream for lack of testing. Or create your own distribution of flawless software and keep it up to date and flawless for the rest of your life.

Re: The modern packager’s security nightmare

#213

Post by drran »

lmm wrote: post #198
bscphil wrote: post #86

Earlier quoted context omitted.

> They should be cut out of the loop. I disagree. This is how you get the Google Play store or the "freeware" app marketplace. It sucks. As a user, I'm quite happy to continue using a traditional distribution even if it means I don't get to use a handful of flashy programs by developers that disagree with the concept of maintainers. So far that choice has been much more positive than negative for me, and I'm doing my…

> If you're creating open source software, they quite literally do! https://www.gnu.org/licenses/gpl-3.0.en.html Clearly you missed the disclaimer of warranty in the licensing terms. Packagers are welcome to maintain their own patches, or their own fork, if they like. But they don't have any right to tell upstream what to do or demand particular guarantees from upstream.

It's temporary, I'm sure. When broken software or a weaponized patch will slip through weak fence of maintainers and then break critical infrastructure, Congress will vote for something to stop that.

Re: The modern packager’s security nightmare

#214

Post by drran »

alfiedotwtf wrote: post #203
Foxboron wrote: post #112

Earlier quoted context omitted.

>With systems like Arch that get us closer and closer to upstream, are maintainers the unnecessary middlemen? Of course they’re not entirely redundant, but maybe a new model of distros like Arch will be more commonplace in the future Arch is an old distribution, very much in the same class as Fedora, Debian, Gentoo and all the traditional ones. What makes you think Arch makes maintainers even slightly redundant? We s…

> We still deal with security issues I don’t see your point. In fact, you’ll most likely find that distros that follow upstream closer than “slow and stable releases” will get their patches as soon as upstream fixes them

> will get their patches as soon as upstream fixes them

i.e. never. Lot of upstreams are abandoned with tons of PR's waiting years for somebody to merge in.

Re: The modern packager’s security nightmare

#215

Post by noobermin »

dataflow wrote: post #33

Why do only *nix folks pull their hair out about this? On Windows programs bundle their dependencies all the time (sometimes even as shared libraries! but without the independent update benefits) and hardly anybody loses sleep over it. Heck, users actually like the fact that it minimizes friction. Nobody claims it's rock-solid security, but does it need to be? Actually, now that I wrote it above, I think I might have…

People honestly need to push back on the constant focus on security, it's an important concern but it shouldn't make life worse for users, in some ways it is in fact being used this way.

Re: The modern packager’s security nightmare

#216

Post by drran »

cycloptic wrote: post #138
steveklabnik wrote: post #100

Earlier quoted context omitted.

Quoting from the documentation I linked: > Strongly prefer to upstream all patches to LLVM before including them in rustc. That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can. But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written…

Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? Assuming there are tests for these LLVM bugs, and assuming the patches are indeed being merged, wouldn't a CI be able to catch when it is safe for downstream users that want to use upstream LLVM to update their Rust installation?

Rust in Fedora is compiled with system LLVM (libLLVM-11.so). Enjoy.

Re: The modern packager’s security nightmare

#217

Post by noobermin »

evmar wrote: post #8
anotherhue wrote: post #4

Correct me if I'm wrong (really) but isn't static linking mostly a problem for packagers? The less a package maintainer modifies the application the better IMO. If application A and B rely on dependency D, which turns out to have a vulnerability, fixed in D', Then _why_ do we think it is anyone but A & B's developers responsibility to update to D' and distribute patched versions? If the packager tries to do it, the c…

The OP is talking about technical details like static linking and language ecosystems, but if you zoom one level out, this comment correctly pins where the underlying problem actually lies: that the distribution maintainers inject themselves into the development process of all the software they ship. When A/B are not well-maintained upstream, distributions can assist by updating its dependency D, but even when A/B ar…

>but as a user of a distribution I find their maintenance of otherwise unmantained software sometimes helpful.

Sometimes. Often times they usually break the software and leave the users hanging dry wondering why A or B don't work anymore. Literally my experience on gentoo as a user for the last few years.

Re: The modern packager’s security nightmare

#218

Post by progval »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

> If you publish a Python library without pinned dependencies, your code is broken.

> you will be unable to today publish an end-to-end set of commands that will work in 2025

Not necessarily.

Since ~2010 I maintain an application with an unpinned requirements.txt; it doesn't even have version constraints at all.

The only breakages I had were either:

1. when switching from Python 2 to Python 3 (obviously)

2. when a new Python version introduces a bug (but Python is not pinnable anyway)

3. once, when a dependency released a new major version and removed an internal attribute I was using in my tests out of laziness (so that one is entirely on me)

The trick is to only use good libraries, that care about not breaking other people's code.

---

It's also worth noting that it's not your job as a developer to make sure your application can be installed anywhere; it's the packager's job to make sure your app can be installed in their distribution.

And if your users want to use pip (which is kind of the Python equivalent of wget + ./configure + make install) instead of apt/yum/... to get the very latest version of your software, then they should be able to figure out how to fix those issues.

Re: The modern packager’s security nightmare

#219

Post by ece »

orra wrote: post #77
hctaw wrote: post #50

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Until then, I'll keep doing it, because it saves me time and money. I don't really buy that dynamic linking all the things is such a boon to security. But I'll link to this: https://drewdevault.com/dynlib > Not including libc, the only libraries which had "…

> I don't really buy that dynamic linking all the things is such a boon to security. Agreed. It's really not a panacea. When you upgrade a library, you probably want to restart the running applications that depend upon this. Dynamic linking won't save you. The solution is having a graph of your dependencies! IIRC, NixOS gets this right. I don't think Debian's apt did?

There are scripts and daemons that help you determine what needs restarting[1,2]. NixOS installs can go in separate directory prefixes when there are conflicts. For Gentoo and other Linux distributions, maintainers usually won't mark something stable without resolving conflicts, and this usually means sticking to older stable version of libraries until newer versions of libraries are fully supported by all installed packages. This can definitely be more work for maintainers, and as the blog posts says, it's a sisyphean task.

[1] https://manpages.debian.org/buster/debian-goodies/checkresta...

[2] https://github.com/liske/needrestart

Re: The modern packager’s security nightmare

#220

Post by rini17 »

pjmlp wrote: post #200
jauer wrote: post #58

Earlier quoted context omitted.

> Two different communities create two very different kinds of software, which run on the same systems, but are created and distributed in very different ways. This is where the disconnect is coming from. The distro maintainers are coming from a world of multi-user systems where backwards compatibility and updating deps without disturbing a user's workload / forcing them to recompile is paramount. Go (and a fair amou…

Additionally, when you have languages with rich library ecosystems, the OS kind of becomes irrelevant, the platform is the language ecosystem. Just to pick Go as an example (not to be lost discussing VMs and such), it doesn't matter if I am targeting bare metal, Linux, Windows, IBM z/OS, AWS special cloud runtime, whatever. As long as the Go code is the same, and someone has done the low level runtime support, it is…

What if there's another software in another language you want to interoperate with? What if you want to avoid containers with their complexity and dubious security record?
Post reply on HN