Packagers don't know best • Hacker News Board

Re: Packagers don't know best

#31

Post by dvanduzer » Fri, Jun 21, 2013, 11:13 PM UTC

tmoertel wrote: post #20

The packagers actually do know what’s best. What they do makes patches flow faster not only downstream but also upstream. Improvements and fixes get to more people and get to them faster. Unbundling upstream libraries from downstream projects flattens the change-flow network, reducing the time it takes for things to get fixed and for the fixes to propagate. For example, say that project P uses library L and bundles a…

They really don't.

This is part of the reason for the plethora of Linux distributions. Some deployments can afford the rapid pace (and consequent instability) of the short term Ubuntu releases or Fedora. Other deployments really do require the longer term stability of the more methodical Ubuntu LTS releases or CentOS / RHEL.

Any improvement requires change, but not all changes are an improvement.

Re: Packagers don't know best

#32

Post by binarycrusader » Fri, Jun 21, 2013, 11:18 PM UTC

kevingadd wrote: post #18

Nux wrote: post #11

There's also the security factor (that many devs today like to ignore); using shared stuff simplifies it. Maybe packagers don't know best, but neither does this guy.

You say this guy doesn't know better, but given that he's talking about shipping a security sensitive application that relies on custom tuned, tested forks of libraries, how can you say that he's wrong for not wanting his library fork replaced with some arbitrary version on an end-user's machine? How can that possibly be safer? It's certainly nice to be able to take an existing library an app depends on, patch it to…

So the developer wants to reduce their cost of properly engineering and documenting their application's usage of a particular library in exchange for significantly increasing the costs of rebuilding and updating every software package that uses the same libraries onto the OS developer and their customers?

Re: Packagers don't know best

#33

Post by dfc » Fri, Jun 21, 2013, 11:19 PM UTC

Look at this ubuntu erlang package, it depends on 40 other packages, as well. That isn’t even the worst of it, if you type ‘erl’ it tells you to install ‘erlang-base’, which only has a handful of dependencies, none of which are any of these erlang libraries!

That package is a dummy package that depends on erlang-base and the rest of the base erlang platform. You would have to force dpkg to ignore dependencies in order to install erlang without erlang-base. I would love to hear how that happened.

Splitting things up into multiple packages makes distributions easier to manage. One person can take the lead on package-dev while another person can take the lead on package-doc. Splitting things up into multiple smaller packages also makes distributing fixes a lot easier. With a one line fix to one include would you rather send out the entire erlang environment or just the small package that needed the fix?

And yes splitting things up to save storage requirements is most useful for resource constrained devices, not new servers/laptops. But it means that a user who is comfortable with Debian or Fedora on the server/desktop can use their same trusty OS on their next project when the device places serious restrictions on system overhead.

Re: Packagers don't know best

#34

Post by binarycrusader » Fri, Jun 21, 2013, 11:20 PM UTC

snprbob86 wrote: post #26

ochs wrote: post #19

The solution is easy: if you fork a project and it becomes incompatible with the upstream, rename it . How is anyone supposed to discriminate between the two versions if they have the same name? Also, I'd say, if your software needs lots of modified dependencies, you're not communicating with those projects properly. If every single project were to fork every one of their dependencies, the result would be maintenance…

> if you fork a project and it becomes incompatible with the upstream, rename it I agree completely, but I'd like to take your idea further in a direction you likely didn't intend. The fundamental observation of distributed version control systems, in my opinion, is: Every commit is essentially a fork. When you combine these two ideas: 1) fork->rename and 2) change==fork, with the 3) identities & values from FP/Cloju…

Version numbers only appear to be folly because proper engineering discipline has not been applied when managing the stability and/or backwards-compatibility of shared interfaces.

If more developers cared about versioning their software appropriately based on incompatible changes or stability guarantees, it would significantly reduce the costs of maintaining OS software distributions and providing integrated software stacks to users.

Re: Packagers don't know best

#35

Post by dfc » Fri, Jun 21, 2013, 11:21 PM UTC

marssaxman wrote: post #4

jordigh wrote: post #2

The point of not using embedded libraries isn't about saving space. It's about not having several slightly different versions of the same bug spread out across several slightly different versions of the same library. Saving space is just a nice side effect, so why not have that too? The DLL hell problem doesn't exist in a GNU-based system because we have sonames. Windows and Mac OS X don't have those; instead, the so…

And yet, the Mac OS X user experience is so much nicer than the one you get with a GNU-based system; you download an app, it is self contained, it works, end of story. I have been hearing the same old story for years about how dependency-tracking package managers are the right way, and yet that environment continues to have problems, as described in the article; while the supposedly inferior Mac OS X packaging system…

The first thing I do with a new mac is install homebrew.

Re: Packagers don't know best

#36

Post by notacoward » Fri, Jun 21, 2013, 11:23 PM UTC

ochs wrote: post #19

The solution is easy: if you fork a project and it becomes incompatible with the upstream, rename it . How is anyone supposed to discriminate between the two versions if they have the same name? Also, I'd say, if your software needs lots of modified dependencies, you're not communicating with those projects properly. If every single project were to fork every one of their dependencies, the result would be maintenance…

>if your software needs lots of modified dependencies, you're not communicating with those projects properly.

This, a hundred times. The OP wants to bundle modified versions of other people's open-source software as part of their own without feeding the changes upstream properly, and that's just not the right way to do things. Distributions' rules discouraging bundled packages are there because even worse things happen if everyone does that. Sometimes the dependent package has to put off packaging a new release for a particular distro until their dependencies are satisfied, but then it's time to put on big-girl panties and move on. Managing dependencies and reducing version sensitivity are part of a developer's job.

Re: Packagers don't know best

#37

Post by shtylman » Fri, Jun 21, 2013, 11:28 PM UTC

asuffield wrote: post #30

marssaxman wrote: post #4

Earlier quoted context omitted.

And yet, the Mac OS X user experience is so much nicer than the one you get with a GNU-based system; you download an app, it is self contained, it works, end of story. I have been hearing the same old story for years about how dependency-tracking package managers are the right way, and yet that environment continues to have problems, as described in the article; while the supposedly inferior Mac OS X packaging system…

Then one day somebody finds a critical bug in zlib and you need to go and fetch an update for every single application you have ever installed on your system. Then it turns out half of them don't have updates.

This is a fallacy. Just because a bug was exposed in a library you used does not mean your software will be exposed in the same way. Your use of the lib may not even overlap with the bug exposure. And I seriously doubt you carefully comb all the libs you select on a project for "security" before you release something.

Re: Packagers don't know best

#38

Post by tmoertel » Fri, Jun 21, 2013, 11:31 PM UTC

dvanduzer wrote: post #31

tmoertel wrote: post #20

The packagers actually do know what’s best. What they do makes patches flow faster not only downstream but also upstream. Improvements and fixes get to more people and get to them faster. Unbundling upstream libraries from downstream projects flattens the change-flow network, reducing the time it takes for things to get fixed and for the fixes to propagate. For example, say that project P uses library L and bundles a…

They really don't. This is part of the reason for the plethora of Linux distributions. Some deployments can afford the rapid pace (and consequent instability) of the short term Ubuntu releases or Fedora. Other deployments really do require the longer term stability of the more methodical Ubuntu LTS releases or CentOS / RHEL. Any improvement requires change, but not all changes are an improvement.

I'm not sure I was able to get my point across to you. Let me try another approach.

The improvement I'm talking about occurs upsteam of the distributions, even though it is caused by the distributions' packaging policies.

Libraries are upstream from projects, and projects are upstream from distributions. If the distributions discourage projects from bundling libraries, this policy will encourage project developers to talk to the upstream library developers to get desired changes into the libraries, rather than go the customize-and-bundle route. This improved coordination and patch-flow benefits the users of the libraries and the users of the projects, regardless of whether those users rely on any particular distribution to get the software. Users are, as always, still free to pick whatever distribution best suits their preferences, or no distribution at all. Still, they benefit from the distributions' debundling policy.

Re: Packagers don't know best

#39

Post by m-r-a-m » Fri, Jun 21, 2013, 11:34 PM UTC

bcl wrote: post #28

m-r-a-m wrote: post #12

My favorite interesting packaging choice is TeX Live in Fedora 18 [1]. There are about 4500 texlive-* packages (out of around 35000 binary packages in Fedora total). The packagers split up the packages based on upstream metadata. [1] https://bugzilla.redhat.com/show_bug.cgi?id=949626

s/interesting/insane/

It does sound insane to have so many packages, but it also follows Fedora's policy of staying as close to upstream as possible. The entire package building process is automated since TeX Live provides the packaging metadata with the source.

They also include the meta packages [1] so you don't have to install every package individually if disk space is not an issue.

[1] https://fedoraproject.org/wiki/Features/TeXLive#Benefit_to_F...

Re: Packagers don't know best

#40

Post by dragonwriter » Fri, Jun 21, 2013, 11:34 PM UTC

binarycrusader wrote: post #34

snprbob86 wrote: post #26

Earlier quoted context omitted.

> if you fork a project and it becomes incompatible with the upstream, rename it I agree completely, but I'd like to take your idea further in a direction you likely didn't intend. The fundamental observation of distributed version control systems, in my opinion, is: Every commit is essentially a fork. When you combine these two ideas: 1) fork->rename and 2) change==fork, with the 3) identities & values from FP/Cloju…

Version numbers only appear to be folly because proper engineering discipline has not been applied when managing the stability and/or backwards-compatibility of shared interfaces. If more developers cared about versioning their software appropriately based on incompatible changes or stability guarantees, it would significantly reduce the costs of maintaining OS software distributions and providing integrated software…

> Version numbers only appear to be folly because proper engineering discipline has not been applied when managing the stability and/or backwards-compatibility of shared interfaces.

Encoding intelligence (beyond, perhaps, simple sequence) in version numbers for software is fundamentally folly.

Encoding intelligence about compatibility in version numbers of APIs is only folly to the extent that "proper engineering discipline has not been applied when managing the stability and/or backwards-compatibility of shared interfaces."

Confusing what makes sense with software and what makes sense with APIs is as problematic as any other confusion of interface with implementation.