Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The modern packager’s security nightmare

blogs.gentoo.org

Post reply on HN
Print view View original

231–240 of 282 posts

Re: The modern packager’s security nightmare

#231

Post by drran »

pvorb wrote: post #21
posix_me_less wrote: post #19

Here is the solution to this Debian/Gentoo/ packager's dilemma. Packagers realize that their numbers are small and they can't keep up fixing all the modern big software (BS) projects that go against their philosophy. They define the core of the OS that they can keep managing in this traditional way (kernel, basic userland, basic desktop, basic server services, basic libraries, security support for all that). "But peo…

Exactly. When you want to install one of those fast-paced pieces of software on your system, you're often dissatisfied with the old version coming with your OS anyway.

I see no problem to bump versions of a few packages, rebuild them, and install them via system package manager. Are you developer or user? Of course, some software may not work after that, but package manager newer stays in your way, unless you have no idea how to use it.

Re: The modern packager’s security nightmare

#232

Post by lifthrasiir »

drran wrote: post #212
lifthrasiir wrote: post #171

Earlier quoted context omitted.

> As a user, I'm quite happy to continue using a traditional distribution even if it means I don't get to use a handful of flashy programs by developers that disagree with the concept of maintainers. As a user, I'm afraid that you don't seem to be a representative of typical users. I would be happy to use a traditional distribution if they don't break on updates, which is still not the case. It's clear that tradition…

Distributions are not breaking software, they are just distributing broken software. Blame upstream for lack of testing. Or create your own distribution of flawless software and keep it up to date and flawless for the rest of your life.

Oh, sure. Softwares are broken, you can't fix that. Traditional distros just happen to (falsely) believe that they can somehow fix at their level. What we need instead is a distro that is resilient to software breakage, not flawless software. Does my hope look that unreasonable to you?

Re: The modern packager’s security nightmare

#233

Post by lrvick »

bscphil wrote: post #13

Over the last several weeks I was working on an essay about this exact problem, including the connection between static linking and bundling. This one is so well done that I probably won't even publish it. But I'll add this, for people who may not immediately see why this is important. I think that the real danger these new technologies represent is not inherently bad technology, but the possibility of ecosystem dama…

Publish it if for no other reason than to add another item in search results for those researching this issue.

It is one if the biggest house of cards issues in the FOSS ecosystem that few are talking about.

Re: The modern packager’s security nightmare

#234

Post by choeger »

CJefferson wrote: post #55
choeger wrote: post #11

Earlier quoted context omitted.

I think it would help if distributions could snatch a piece of real estate in upstream software. Something like, "in every project, the debian/ root folder belongs to the debian project and follows their rules". The packaging could then verify this folder and put their patches, build scripts, etc. there. This would help upstream communication a lot, I guess.

The problem is there isn't just Debian, there is Debian and arch and Gentoo and nix and guix and freebsd and cygwin (at least). Then, how do I check who should have merge access to all these directories?

This organizational aspect could be outsourced to one dedicated organization. Not all distributions have to join. I think if 20% did, 80% of the problems would be solved.

Re: The modern packager’s security nightmare

#235

Post by caseyavila »

My goodness, It’s as if [the bundling part of] this post speaks directly to me. I use Gentoo as my daily driver, and school forces us to use Zoom for classes. Like most other proprietary software vendors, Zoom chooses to bundle a great deal of shared objects along with their Linux binary. Gentoo however, chooses to let Zoom use the system libraries instead of those bundled. Through some sort of ABI incompatibility, Zoom’s use of my system’s libraries causes joining a class to get stuck on a “Connecting...” screen. It doesn’t happen every time, but it happens often enough to annoy my teachers, who end up having to let me in multiple times.

I don’t really want to run Zoom’s precompiled libraries, and the version numbers of the objects match one to one. But alas, Zoom probably modified the libraries to fit their own applications needs. I can only hope that a future update of Zoom magically solves my problems.

Re: The modern packager’s security nightmare

#236

Post by josephg »

drran wrote: post #207
josephg wrote: post #161

Earlier quoted context omitted.

> The "problem", in the sense that it's a problem, is that these languages and related technologies are all pretty good! Yes; and frankly the development ecosystem for making software for linux & friends on top of apt/etc is terrible - at least from the perspective of a modern professional software engineer. The assumption is C, and C programs have no package manager - so of course dependencies get bundled / vendored…

So, you deliberately chose distro intended for users, with low version churn, instead of distro intended for developers, e.g. Fedora, which ships even pre-release versions sometimes, and now you blame ... apt? Just curious, what you are using for coding? MS Word or Excel? For example, Linus uses Fedora/MATE/Emacs. It's relatively easy to convert between packages between different packagers/distros. Automatic converte…

If you want your software to work everywhere, you either need to take responsibility for making it build everywhere (yes, including old versions of Debian which don’t support the versions of your dependencies you need) or you punt that work to someone else - in which case your software simply won’t work on lots of computers. From the perspective of an upstream maintainer, the status quo is pretty awful.

There’s a reason people are turning to docker - because the “portable executable” idea on Linux is so often broken by weird incompatibilities between libc versions or some important dependency being missing or broken from some users’ systems. Automatic package translation suffers the exact same problem - a dynamically linked binary you build on your computer often won’t run on my computer.

Re: The modern packager’s security nightmare

#237

Post by josephg »

zrm wrote: post #170
josephg wrote: post #161

Earlier quoted context omitted.

> The "problem", in the sense that it's a problem, is that these languages and related technologies are all pretty good! Yes; and frankly the development ecosystem for making software for linux & friends on top of apt/etc is terrible - at least from the perspective of a modern professional software engineer. The assumption is C, and C programs have no package manager - so of course dependencies get bundled / vendored…

> This worked in a world when there wasn't much software, when releases were rare and when most programs only had one or two dependencies. None of these properties are true any more. I don't think this is even the problem. It's that upstream maintainers have stopped worrying about compatibility. Once upon a time you would have regular minor releases of some package. 3.0.2, 3.0.3, 3.0.4, but they were all backwards co…

Interesting - I didn’t know that. Can you give some examples?

Re: The modern packager’s security nightmare

#238

Post by virgo_eye »

bscphil wrote: post #13

Over the last several weeks I was working on an essay about this exact problem, including the connection between static linking and bundling. This one is so well done that I probably won't even publish it. But I'll add this, for people who may not immediately see why this is important. I think that the real danger these new technologies represent is not inherently bad technology, but the possibility of ecosystem dama…

The distro model did not work, which is why the other model took over.

I used to try to religiously follow the recommendation only to install Python packages which had been repackaged by Debian. Fine, but it meant that you couldn't use any even slightly obscure package, nor one that was younger than some timelag, which ranged from a few months to a couple of years.

Inevitably, you want to pip install something. Then the repercussions of mixing Debian packages and pip packages are a whole new set of problems. And you can't get anyone to look at your problem, even if it's a common issue which the Debian packager could fix or workaround, because 'pip is not supported, you should install this via `apt install python-foo`'.

The best solution is and was to only use pip packages, along with some form of isolation from the wider system, whether virtual environments, containers or what. Python now has extremely good native tools to work this way, and so do most modern languages. I only work like this now, and so, it appears, do all the maintainers of my dependencies and my transitive dependencies. Development cycles are much faster, and it all just works.

Re: The modern packager’s security nightmare

#239

Post by virgo_eye »

bscphil wrote: post #47
hctaw wrote: post #42

Earlier quoted context omitted.

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want to see it stick around. I disagree, it's proven to be inadequate for modern software development and that's why these new languages/ecosystems are springing up. The least reliable way to package and distribute software is by relying on traditional package manag…

> I disagree, it's proven to be inadequate for modern software development Well, that's exactly why the OP (and my essay) are "anti" modern software development in many ways. The view is that we're moving away from the traditional open source ecosystem and methods of software development with these new technologies, which (to be clear) are good technologies, but were created mostly to solve problems that some large c…

Why are you comparing distro package managers to the Play store or the Chrome extension store? You should be comparing them to npm, pypi, etc. That's clearly the context of this discussion. No one is saying that the Chrome extension store does a better job than package managers. The language ecosystems do an amazingly better job.

Re: The modern packager’s security nightmare

#240

Post by virgo_eye »

ticviking wrote: post #79
oblio wrote: post #56

Earlier quoted context omitted.

The open source solutions are primarily for C/C++. A bit for Perl, a bit for Python. But they haven't really moved on. Java has been tacked on since forever. Same for .NET, JavaScript, whatever. And if you wanted your program propagated to all major distros you'd have to wait a decade. Nobody has time for that. Not corporations, not mom and pop stores, and I doubt many hobbyists.

Then you always have the option to package your software for the ones you use and not worry about the rest. If you want to make your program popular then packaging is part of the process needed.

I mean the whole argument here is that distro packaging is not needed, that there's a better way of doing it.
Post reply on HN