Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The modern packager’s security nightmare

blogs.gentoo.org

Post reply on HN
Print view View original

91–100 of 282 posts

Re: The modern packager’s security nightmare

#91

Post by gpm »

AshamedCaptain wrote: post #64
syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

> In short unpinned dependencies mean hopeless bit-rot. No, this is not true, for the simple reason that there will _always_ be unpinned dependencies (e.g. your compiler. your hardware. your processor) and thus _those_ are the ones that will guarantee bitrot. Pinning a dependency only _guarantees you rot the same or even faster_ because now it's less likely that you can use an updated version of the dependency that s…

> your compiler

Compilers of languages like C, C++, Rust, Go etc go above and beyond to maintain backwards compatibility. It is extremely likely that you will still be able to compile old code with a modern compiler.

> your processor

Hardware is common enough that people go out of their way to make backwards compatibility shims. Things like rosetta, qemu, all the various emulators for various old gaming systems, etc.

> your hardware

Apart from your CPU (see above) your hardware goes through abstraction layers designed to maintain long term backwards compatibility. Things like opengl, vulkan, metal, etc. The abstraction layers are in widespread enough use that as older ones stop being outdated people start implementing them on top of the newer layers. E.g. here is OpenGL on top of Vulkan: https://www.collabora.com/news-and-blog/blog/2018/10/31/intr...

> [Your kernel]

Ok, you didn't say this part, but it's the other big unpinned dependency. And it too goes above and beyond to maintain backwards compatibility. In fact Linus has a good rant on nearly this exact topic that I'd recommend watching: https://www.youtube.com/watch?v=5PmHRSeA2c8&t=298s

> Pinning a dependency only _guarantees you rot the same or even faster_ because now it's less likely that you can use an updated version of the dependency that supports more recent hardware.

Dependencies are far more likely to rot because they change in incompatible ways than the underlying hardware does, even before considering emulators. It's hard to take this suggestion seriously at all.

Re: The modern packager’s security nightmare

#92

Post by giantrobot »

eecc wrote: post #44
giantrobot wrote: post #29

Earlier quoted context omitted.

We should call this newly invented and wholly original concept a "container". The software gets "contained". It just might work. /s

You mean cgroups, or zones don’t you? Docker (was, last time I heard) a security disaster, not generating robust layer hashes, lacking user isolation, and plenty just running as root...

There's more to containers on Linux than just Docker.

Re: The modern packager’s security nightmare

#93

Post by AshamedCaptain »

hctaw wrote: post #50

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Until then, I'll keep doing it, because it saves me time and money. I don't really buy that dynamic linking all the things is such a boon to security. But I'll link to this: https://drewdevault.com/dynlib > Not including libc, the only libraries which had "…

> But I'll link to this: https://drewdevault.com/dynlib

We already discussed this in HN https://news.ycombinator.com/item?id=23654353 . Both data and conclusions are utterly suspect.

Re: The modern packager’s security nightmare

#94

Post by Rusky »

archi42 wrote: post #89
steveklabnik wrote: post #24

There is a lot I could say about this article, but I've kinda been in this whole Texas natural disaster situation. But I do think it's worth pointing out one thing: > Rust bundles a huge fork of LLVM It is not super huge, and we try to upstream patches regularly to minimize the fork. We also regularly re-base all current patches on each release of LLVM, so it's less 'fork' and more 'maintain some extra commits to fix…

What's the problem with focusing on upstreaming the patches and getting rid of rust's "staging fork"? Especially if they fix something as serious as miscompilations, wouldn't those patches be VERY much needed upstream? I'm asking out of genuine interest.

That's already done. The point is that it takes time, and there are always more of them, so at any point in time there are likely to be a handful of them still in-process of moving upstream.

Re: The modern packager’s security nightmare

#95

Post by AshamedCaptain »

ampdepolymerase wrote: post #25
rosshemsley wrote: post #22

Whilst I don't totally disagree with many of the points here, I think there's a wider picture to many of these issues. The author is concerned with installing packages on user machines: which are typically very long-lived installs - maybe a user has the same machine with the same dependencies for years. However, for many engineers, (such as myself), a binary may not be used past even a few days from when it was first…

The way iOS and Fuchsia are dealing with the problem is to completely lockdown the operating system with a tight permissions system. An app can be compromised but the damage is limited. Perhaps it is time for servers to move to a similar model.

> An app can be compromised but the damage is limited

AKA the "we don't care" security model. What exact use is the fact that the web browser is "contained" if it is compromised? The mail client? Your PIM program? On a server, what use is that the database engine is contained if it is compromised?

I am the first to accept the security benefits of sandboxing, but it is just _one_ thing. It doesn't even help against the majority of issues. Not even on Android/iOS.

Re: The modern packager’s security nightmare

#97

Post by kuschku »

AshamedCaptain wrote: post #64
syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

> In short unpinned dependencies mean hopeless bit-rot. No, this is not true, for the simple reason that there will _always_ be unpinned dependencies (e.g. your compiler. your hardware. your processor) and thus _those_ are the ones that will guarantee bitrot. Pinning a dependency only _guarantees you rot the same or even faster_ because now it's less likely that you can use an updated version of the dependency that s…

> No, this is not true, for the simple reason that there will _always_ be unpinned dependencies (e.g. your compiler. your hardware. your processor) and thus _those_ are the ones that will guarantee bitrot.

Docker with sha256 tags fixes that issue (and docker container even specify a processor architecture).

Re: The modern packager’s security nightmare

#98

Post by AshamedCaptain »

gpm wrote: post #91
AshamedCaptain wrote: post #64

Earlier quoted context omitted.

> In short unpinned dependencies mean hopeless bit-rot. No, this is not true, for the simple reason that there will _always_ be unpinned dependencies (e.g. your compiler. your hardware. your processor) and thus _those_ are the ones that will guarantee bitrot. Pinning a dependency only _guarantees you rot the same or even faster_ because now it's less likely that you can use an updated version of the dependency that s…

> your compiler Compilers of languages like C, C++, Rust, Go etc go above and beyond to maintain backwards compatibility. It is extremely likely that you will still be able to compile old code with a modern compiler. > your processor Hardware is common enough that people go out of their way to make backwards compatibility shims. Things like rosetta, qemu, all the various emulators for various old gaming systems, etc.…

> Dependencies are far more likely to rot because they change in incompatible ways than the underlying hardware does

Yes, that is true. It is also very likely that you can more easily go back to a previous version of a dependency than you can go back to a previous hardware. The argument is that, therefore, pinning can only speed up your rotting.

If you don't statically link your dependencies, and due to an upgrade something breaks, you can always go back to the previous version. If you statically link, and the hardware, compiler, processor, operating system, whatever causes your software to break, now you can't update the dependency that is causing the breakage. And it is likely that your issue is within that dependency.

Pinning can only make you rot faster.

Re: The modern packager’s security nightmare

#99

Post by Qwertious »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

>Of course this cannot work. _Think about it_. There are innumerable ways two pieces of code can be incompatible.

There's a very simple solution here: just don't write bugs.

Re: The modern packager’s security nightmare

#100

Post by steveklabnik »

archi42 wrote: post #89
steveklabnik wrote: post #24

There is a lot I could say about this article, but I've kinda been in this whole Texas natural disaster situation. But I do think it's worth pointing out one thing: > Rust bundles a huge fork of LLVM It is not super huge, and we try to upstream patches regularly to minimize the fork. We also regularly re-base all current patches on each release of LLVM, so it's less 'fork' and more 'maintain some extra commits to fix…

What's the problem with focusing on upstreaming the patches and getting rid of rust's "staging fork"? Especially if they fix something as serious as miscompilations, wouldn't those patches be VERY much needed upstream? I'm asking out of genuine interest.

Quoting from the documentation I linked:

> Strongly prefer to upstream all patches to LLVM before including them in rustc.

That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can.

But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain.

So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream.

(EDIT: Also, what Rusky said: there's basically always going to be more than one patch, in various stages of this process...)

Post reply on HN