Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The modern packager’s security nightmare

blogs.gentoo.org

Post reply on HN
Print view View original

261–270 of 282 posts

Re: The modern packager’s security nightmare

#261

Post by Nixinova »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

If you're introducing breaking changes in every new release you should still be in the 0.x stage of SemVer. You're doing something wrong if you end up on v77.0.0. The Node ecosystem's strict compliance with SemVer works fine 99% of the time because SemVer is indeed an effective versioning system (when people use it right).

Re: The modern packager’s security nightmare

#262

Post by hedora »

orra wrote: post #77
hctaw wrote: post #50

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Until then, I'll keep doing it, because it saves me time and money. I don't really buy that dynamic linking all the things is such a boon to security. But I'll link to this: https://drewdevault.com/dynlib > Not including libc, the only libraries which had "…

> I don't really buy that dynamic linking all the things is such a boon to security. Agreed. It's really not a panacea. When you upgrade a library, you probably want to restart the running applications that depend upon this. Dynamic linking won't save you. The solution is having a graph of your dependencies! IIRC, NixOS gets this right. I don't think Debian's apt did?

Apt has a graph of dependencies and anti-dependencies. They’ve been getting this right since the late 90s.

Re: The modern packager’s security nightmare

#263

Post by hedora »

syllogism wrote: post #20

This will be a somewhat intemperate response, because as a developer of a significant library I found this quite irritating. If you publish a Python library without pinned dependencies, your code is broken. It happens to work today, but there will come a day when the artifact you have published no longer works. It's only a matter of time. The command the user had run before, like "pip install spacy==2.3.5" will no lo…

I get the impression that this advice is accurate for the python ecosystem, but that’s because the entire ecosystem is broken with respect to backwards compatibility.

The exact same mechanisms work fine with other programming languages, and (more importantly, probably) different developer communities.

In fairness, Python’s lack of static types does make things worse than the situation for compiled languages. (Though that’s a general argument against writing non-throwaway code in python).

People claim node does better, even though JS is also missing static types, so presumably they solved this issue somehow (testing, maybe?). I don’t use it, so I have no idea.

Re: The modern packager’s security nightmare

#264

Post by pvorb »

drran wrote: post #231
pvorb wrote: post #21

Earlier quoted context omitted.

Exactly. When you want to install one of those fast-paced pieces of software on your system, you're often dissatisfied with the old version coming with your OS anyway.

I see no problem to bump versions of a few packages, rebuild them, and install them via system package manager. Are you developer or user? Of course, some software may not work after that, but package manager newer stays in your way, unless you have no idea how to use it.

I'm a software developer and a user of package managers. A package manager that allows me to only have one version of a software package installed will inevitably get in my way if I need two programs that need different, incompatible versions of that dependency. It happens quite a lot in my experience.

Re: The modern packager’s security nightmare

#265

Post by drran »

pvorb wrote: post #264
drran wrote: post #231

Earlier quoted context omitted.

I see no problem to bump versions of a few packages, rebuild them, and install them via system package manager. Are you developer or user? Of course, some software may not work after that, but package manager newer stays in your way, unless you have no idea how to use it.

I'm a software developer and a user of package managers. A package manager that allows me to only have one version of a software package installed will inevitably get in my way if I need two programs that need different, incompatible versions of that dependency. It happens quite a lot in my experience.

Basically, you want to put two versions of the same file(s) into one file.

If you need two versions of almost same set of files, then chose different name for the package, e.g. package-2, chose different base directory for package files, e.g. /usr/share/package-2, and chose different names for binaries, e.g. /usr/bin/binary-2. It's not a magic. Just look at examples, e.g. python2 and python3.

Re: The modern packager’s security nightmare

#266

Post by nonameiguess »

forrestthewoods wrote: post #101
sigjuice wrote: post #73

Earlier quoted context omitted.

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Maybe do the trendy thing and link your whole distro together with your app (a.k.a containers)?

The fact that containers exist and are prevalent is a damning indictment on the Linux model imho. Producing software that can simply launch is so wildly complex that the solution is... to snapshot an entire operating system install to run the application. That’s INSANE. But it’s unfortunately required. https://xkcd.com/1987/ > Now, for the worst of all — one that combines all the aforementioned issues, and adds even…

I don't think this is a fair characterization of why containers are a killer app. Full-OS virtualization was a killer app for hosting providers because it enabled multi-tenancy, allowing you to oversubscribe physical resources knowing that most applications you host aren't going to be experiencing anywhere near peak load 100% of the time. Containers allow you to take this a step further by allowing multiple applications with different versions of the same dependencies to run together on the same physical or virtual server without needing to worry about symbol clashes and without having to install one application's dependencies into some different than expected location while manipulating LD_LIBRARY_PATH and PATH.

This solved a huge problem for us in the geointelligence ground processing community. The various product generation algorithms tend to all be native code, but they're produced by different contractors, on different delivery schedules, some of whom are no longer on contract at all to provide updates. What do you do when four of them all depend on libxerces-c but all four depend on a different version? Tell them to stop pinning dependencies and update? How, if they're not on contract to do it? Tell the US Congress to get off their butts and award more money to the backing agencies so they can get their developers back on contract? Good luck. In practice, what we ended up doing, without wanting to stripe certain applications to only run on specific servers when some of them are used a ton more than others, was to install dependencies on an NFS mount shared by all the fast compute hosts, into application-specific subdirectories, and then set PATH and LD_LIBRARY_PATH before calling the executable that did all the work.

This system was terrible! But before containers, it was the best we could do. Now we just deploy each individual processing algorithm in its own container, and they can depend on whatever they want, all using standard system path under the illusion they have an entire OS and filesystem all to themselves. and never clash with each other.

Re: The modern packager’s security nightmare

#267

Post by Chyzwar »

pvorb wrote: post #264
drran wrote: post #231

Earlier quoted context omitted.

I see no problem to bump versions of a few packages, rebuild them, and install them via system package manager. Are you developer or user? Of course, some software may not work after that, but package manager newer stays in your way, unless you have no idea how to use it.

I'm a software developer and a user of package managers. A package manager that allows me to only have one version of a software package installed will inevitably get in my way if I need two programs that need different, incompatible versions of that dependency. It happens quite a lot in my experience.

To some extent you can do it with snap. https://snapcraft.io/blog/parallel-installs-test-and-run-mul...

A lot of discussion here is already solved by snap/flatpack.

Re: The modern packager’s security nightmare

#268

Post by bscphil »

pdonis wrote: post #175
hctaw wrote: post #42

Earlier quoted context omitted.

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want to see it stick around. I disagree, it's proven to be inadequate for modern software development and that's why these new languages/ecosystems are springing up. The least reliable way to package and distribute software is by relying on traditional package manag…

>I disagree, it's proven to be inadequate for modern software development You're not disgreeing with the post you responded to; you're just stating a different priority. hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software . That means trustworthy to the user . It says nothing at all about how hard producing that software is for the developer. You are saying that the di…

I agree with your comment, but to be clear,

> hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software

I think you mean I (bscphil) said that. hctaw was the user disagreeing, saying it makes things harder for developers.

Re: The modern packager’s security nightmare

#269

Post by pdonis »

bscphil wrote: post #268
pdonis wrote: post #175

Earlier quoted context omitted.

>I disagree, it's proven to be inadequate for modern software development You're not disgreeing with the post you responded to; you're just stating a different priority. hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software . That means trustworthy to the user . It says nothing at all about how hard producing that software is for the developer. You are saying that the di…

I agree with your comment, but to be clear, > hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software I think you mean I (bscphil) said that. hctaw was the user disagreeing, saying it makes things harder for developers.

> I think you mean I (bscphil) said that.

Oops, yes, you're right. Sorry for the mixup.

Re: The modern packager’s security nightmare

#270

Post by microtonal »

Foxboron wrote: post #119
dralley wrote: post #116

Earlier quoted context omitted.

Maybe they have heard of the AUR and think it's how the rest of Arch works?

I'd be terrified, but that is how NixOS works.

That is nonsense and you seem to spread this misinformation in a lot of places. You should also add a disclaimer that you are part of the Arch team.

AUR: Anyone can create an account and upload PKGBUILDs. There are no checks at all. AUR users should verify whether PKGBUILDs are not malicious. In practice, a lot of people use things like yaourt to install packages from the AUR without verifying the PKGBUILDs.

nixpkgs: anyone can contribute a PR with a new package, package update, or package modification. However, changes only get added to nixpkgs after someone with commit privileges verifies the PR and merges it. Also, a common misconception is that nixpkgs package maintainers can merge changes. This is false, only a much smaller set of committers can merge changes in the actual nixpkgs repository.

nixpkgs is more like the Arch Community repository, where committers are long-time contributors with a track record of high-quality contributions. Parts of nixpkgs are like Arch Core/Extra, because they are marked using the GitHub codeowners mechanism and changes are generally not merged unless approved through the code owners.

Disclaimer: I am a nixpkgs committer, former Arch user and AUR contributor.

Post reply on HN