Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The modern packager’s security nightmare

blogs.gentoo.org

Post reply on HN
Print view View original

51–60 of 282 posts

Re: The modern packager’s security nightmare

#51

Post by nextos »

hctaw wrote: post #42
bscphil wrote: post #13

Over the last several weeks I was working on an essay about this exact problem, including the connection between static linking and bundling. This one is so well done that I probably won't even publish it. But I'll add this, for people who may not immediately see why this is important. I think that the real danger these new technologies represent is not inherently bad technology, but the possibility of ecosystem dama…

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want to see it stick around. I disagree, it's proven to be inadequate for modern software development and that's why these new languages/ecosystems are springing up. The least reliable way to package and distribute software is by relying on traditional package manag…

Some of the things you mention are not incompatible with package managers. Have you considered Nix?

I actually agree with the parent post in terms of strongly preferring package managers to other means of distributing software. I've always found Linux much easier to admin than other OSes simply because of package managers.

Re: The modern packager’s security nightmare

#52

Post by charliesome »

choeger wrote: post #14

Would it be feasible for rust to ship "re-link" scripts? That is, when a cargo dependency gets updated, the distribution can check whether packages code needs to be re-linked? Similarly, when distributed code (say libssl) changes, the rust packages can be re-linked by the distribution on-site?

You can achieve this by recompiling the software. Cargo will not recompile dependencies that have not changed.

Re: The modern packager’s security nightmare

#53

Post by anderskaseorg »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

If you publish a Python library with pinned dependencies, your code is broken as soon as someone tries to use it with another Python library with pinned dependencies, unless you happened to pin exactly the same version of the dependencies you have in common.

Python libraries should not pin dependencies. _Applications_ can pin dependencies, including all recursive dependencies of their libraries. There are tools like Pipenv and Poetry to make that easy.

This is less of an issue in (say) Node.js, where you can have multiple different versions of a library installed in different branches of the dependency tree. (Though Node.js also has a strong semver culture that almost always works well enough that pinning exact versions isn’t necessary.)

Re: The modern packager’s security nightmare

#54

Post by acidbaseextract »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

It's maddening to hear people say things like, "Oh if everyone just used semantic versioning this wouldn't be a problem". Of course this cannot work. _Think about it_. There are innumerable ways two pieces of code can be incompatible. ... If you call these things "breaking" changes, you will constantly be increasing the major version.

One of the things that prompted the OP was this breakage in Python's cryptography package [1] (OP actually opened this issue) due to the introduction of a Rust dependency in a 0.0.x release. The dependency change didn't change the public API at all, but did still cause plenty of issues downstream. It's a great question on the topic of semver to think about how to handle major dependency changes that aren't API changes. Personally, I would have preferred a new major release, but that's exactly your point syllogism — it's a matter of opinion.

As a sidenote, Alex Gaynor, one of the cryptography package maintainers is on a memory-safe language crusade. Interesting to see how that crusade runs into conflict with the anti-static linking crusade that distro packagers are on. I find both goals admirable from a security perspective. This stuff is hard.

[1] https://github.com/pyca/cryptography/issues/5771

Re: The modern packager’s security nightmare

#55

Post by CJefferson »

choeger wrote: post #11
evmar wrote: post #8

Earlier quoted context omitted.

The OP is talking about technical details like static linking and language ecosystems, but if you zoom one level out, this comment correctly pins where the underlying problem actually lies: that the distribution maintainers inject themselves into the development process of all the software they ship. When A/B are not well-maintained upstream, distributions can assist by updating its dependency D, but even when A/B ar…

I think it would help if distributions could snatch a piece of real estate in upstream software. Something like, "in every project, the debian/ root folder belongs to the debian project and follows their rules". The packaging could then verify this folder and put their patches, build scripts, etc. there. This would help upstream communication a lot, I guess.

The problem is there isn't just Debian, there is Debian and arch and Gentoo and nix and guix and freebsd and cygwin (at least).

Then, how do I check who should have merge access to all these directories?

Re: The modern packager’s security nightmare

#56

Post by oblio »

bscphil wrote: post #47
hctaw wrote: post #42

Earlier quoted context omitted.

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want to see it stick around. I disagree, it's proven to be inadequate for modern software development and that's why these new languages/ecosystems are springing up. The least reliable way to package and distribute software is by relying on traditional package manag…

> I disagree, it's proven to be inadequate for modern software development Well, that's exactly why the OP (and my essay) are "anti" modern software development in many ways. The view is that we're moving away from the traditional open source ecosystem and methods of software development with these new technologies, which (to be clear) are good technologies, but were created mostly to solve problems that some large c…

The open source solutions are primarily for C/C++. A bit for Perl, a bit for Python. But they haven't really moved on. Java has been tacked on since forever. Same for .NET, JavaScript, whatever.

And if you wanted your program propagated to all major distros you'd have to wait a decade.

Nobody has time for that. Not corporations, not mom and pop stores, and I doubt many hobbyists.

Re: The modern packager’s security nightmare

#57

Post by loloquwowndueo »

ampdepolymerase wrote: post #25
rosshemsley wrote: post #22

Whilst I don't totally disagree with many of the points here, I think there's a wider picture to many of these issues. The author is concerned with installing packages on user machines: which are typically very long-lived installs - maybe a user has the same machine with the same dependencies for years. However, for many engineers, (such as myself), a binary may not be used past even a few days from when it was first…

The way iOS and Fuchsia are dealing with the problem is to completely lockdown the operating system with a tight permissions system. An app can be compromised but the damage is limited. Perhaps it is time for servers to move to a similar model.

Just install all your software from snaps on an Ubuntu system ;) (just kidding, snaps have a whole bunch of issues for server software )

Re: The modern packager’s security nightmare

#58

Post by jauer »

bscphil wrote: post #13

Over the last several weeks I was working on an essay about this exact problem, including the connection between static linking and bundling. This one is so well done that I probably won't even publish it. But I'll add this, for people who may not immediately see why this is important. I think that the real danger these new technologies represent is not inherently bad technology, but the possibility of ecosystem dama…

> Two different communities create two very different kinds of software, which run on the same systems, but are created and distributed in very different ways.

This is where the disconnect is coming from. The distro maintainers are coming from a world of multi-user systems where backwards compatibility and updating deps without disturbing a user's workload / forcing them to recompile is paramount.

Go (and a fair amount of rust/python work) come from the land of CI/CD and, to a lesser extent, monorepos. When you are rebuilding the world above a bare minimum of the OS literally on every commit (or at least several times per day), it's easier to reason about code that is running if you can look at the commit that a bin was built from and know exactly what's inside (including all deps).

Re: The modern packager’s security nightmare

#59

Post by delroth »

posix_me_less wrote: post #19

Here is the solution to this Debian/Gentoo/ packager's dilemma. Packagers realize that their numbers are small and they can't keep up fixing all the modern big software (BS) projects that go against their philosophy. They define the core of the OS that they can keep managing in this traditional way (kernel, basic userland, basic desktop, basic server services, basic libraries, security support for all that). "But peo…

You're making a distinction between "packagers" and "users" which does not exist. Packagers are advanced users that take the initiative to improve their distro when they find software they want to use and that isn't integrated in their distro.

> Let the new complicated juggernaut software be deployed and configured by the users, according to developers' instructions, using whatever modern mechanism they prefer

I'm not sure what "they" refers to here (developers or users). The existence of packagers is proof that some subset of users "prefer" that software "be deployed and configured" via distro package managers.

Re: The modern packager’s security nightmare

#60

Post by oblio »

dataflow wrote: post #40
GartzenDeHaes wrote: post #36

Earlier quoted context omitted.

The unix culture comes from a shared multi-user perspective, whereas the windows point of view tends to be towards a single user desktop. In a shared environment, it's not acceptable for applications to change shared system components. Although this is maybe less important today, the culture still persists.

> In a shared environment, it's not acceptable for applications to change shared system components. That era is long gone on Windows (literally since XP I think). In fact I've had to do this far more on Linux than on Windows in recent memory... the latest instance ironically being the glibc CLOCK_REALTIME mess on WSL. Right now programs bundle whatever they need, and these go into the app directory (like \Program Fil…

You're misreading his comment. He's saying Windows is share nothing.
Post reply on HN