Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#191

Post by tremon »

steveklabnik wrote: post #109
dboreham wrote: post #99

Earlier quoted context omitted.

Go and Rust don't have packages. They have tooling to pull library code from git and build it in-place to be linked into a local project. That's not (really) packaging.

Cargo does not “pull library code from git” unless you expressly ask for it to. And given that packages have to depend on other packages, and cannot depend on a git repository, that feature is mostly useful for testing bug fixes, private repos for leaf packages, stuff like that.

I thought Rust doesn't have binary libraries? If you declare a Rust dependency, cargo pulls its source and builds it together with your own code? I assume that's what the GP meant, anyway.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#192

Post by tremon »

jmull wrote: post #147
qbasic_forever wrote: post #117

There is no real theory because it's a solved problem--packages and dependencies form a DAG or graph and it's just an ordering and traversal problem that's easily solved. The pain and fragmentation you're mentioning is that everyone has different opinions about how they want to configure/bundle/organize code and package metadata. That's really the only core difference between deb/rpm, npm/yarn, pip/poetry/conda/ , ha…

Well, not DAG, because the A is for acyclic, and dependency graphs can definitely contain cycles.

In the case of immutable builds (and hence immutable dependencies), dependency graphs cannot contain cycles. At most they can have the same package occur twice in the graph, but with different dependencies or different build options.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#193

Post by samsquire »

I think the problem is the answer to this question:

What is harder regarding computation - arranging for a computation or doing the computation?

What I mean is that the computation of an addition or subtraction is easy but the arranging of instructions of code into an algorithm is hard. Everything has to be in the right place for the algorithm to work right.

This is why packaging is so difficult. Each algorithm and code expects things to be in a certain place. ABIS and packaging for them is tedious.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#194

Post by steveklabnik »

tremon wrote: post #191
steveklabnik wrote: post #109

Earlier quoted context omitted.

Cargo does not “pull library code from git” unless you expressly ask for it to. And given that packages have to depend on other packages, and cannot depend on a git repository, that feature is mostly useful for testing bug fixes, private repos for leaf packages, stuff like that.

I thought Rust doesn't have binary libraries? If you declare a Rust dependency, cargo pulls its source and builds it together with your own code? I assume that's what the GP meant, anyway.

Source code is stored in an S3 bucket owned by crates.io, and not from any form of source control, including git.

That the entire code of all dependencies is hosted in one place is an important differentiator of crates.io vs what Go does, which is why this is relevant in this context. Crates.io is centralized, with all of the advantages and disadvantages that that brings, and Go is decentralized, with all of the advantages and disadvantages that brings.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#195

Post by noahtallen »

eurasiantiger wrote: post #80
xiphias2 wrote: post #23

Earlier quoted context omitted.

Npm has lots of not understandable mistakes: it should already be as good with ES6+TS as other systems. Import should just work everywhere. We should have left Commonjs a long time ago, while keeping backwards compatibility. At the same time what I see with the node+npm system is that everything is just ,,it just doesn't work by default''. Having 10 other package managers doesn't work either, they are faster, but don…

Just use .mjs extension for your code, add extensions to your imports and everything does just work.

Except in typescript, where you have to import a file with the .js extension, despite the actual file being .ts: https://github.com/microsoft/TypeScript/issues/42151, https://github.com/microsoft/TypeScript/issues/49083, https://github.com/microsoft/TypeScript/issues/16577

This one issue (IMO) is preventing parts of the ecosystem from switching to esm.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#196

Post by pxc »

tremon wrote: post #190
pxc wrote: post #176

Earlier quoted context omitted.

> It's been there for 15 or 20 years. Has it? What I remember being the standard practice for customizing packages was apt-get source && sudo apt-get build-dep && vim && debuild -b fewer than 15 years ago. And it's not much like what the parent poster has in mind. It modifies the global state of your system, it requires multiple steps, it produces a different artifact than you might get in the repos at any given time…

With Nix or Guix, when a cached binary artifact for a given package is missing, you may not even notice, the fallback to building from source is so transparent. I consider this to be one of the main drawbacks of these systems -- it assumes that every machine is build-capable. I do not want compilers installed on my production servers (nor would I want my RPi to attempt to rebuild half the world because the package se…

> it assumes that every machine is build-capable

It doesn't, fortunately. You can tell Nix to rely exclusively on the binary cache, or configure it to have 0 build threads locally and use remote builders.

For a server deployment, many popular choices don't have the server build anything, and in some cases it may not even have any users who are allowed to invoke `nix-*` or anything.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#197

Post by goodpoint »

pxc wrote: post #176
goodpoint wrote: post #159

Earlier quoted context omitted.

> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

> It's been there for 15 or 20 years. Has it? What I remember being the standard practice for customizing packages was apt-get source && sudo apt-get build-dep && vim && debuild -b fewer than 15 years ago. And it's not much like what the parent poster has in mind. It modifies the global state of your system, it requires multiple steps, it produces a different artifact than you might get in the repos at any given time…

Please do not assume I don't know what Nix and Guix do.

> you'd understand why this is a silly point to raise

It isn't.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#198

Post by haradion »

otabdeveloper4 wrote: post #184
pxc wrote: post #180

Earlier quoted context omitted.

Pretend I asked the nice version of this, for the benefit of other readers? I do think a Nix-like tool to be used for writing/replacing package managers for new programming languages would benefit from some dependency resolving functionality, instead of just saying 'shove it in a monorepo or pin via VCS refs instead of semver'. Is that not necessary?

Nix is capable of that, but you need a database mapping versions to VCS refs. Nixpkgs doesn't go down that road because it would be unwieldy at their scale, but more specialized nix-based tools do.

At least for (unlocked) flake sources, you could probably use a branch naming convention instead of a separate database, although that requires your repo to be set up a specific way.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#199

Post by sixstringtheory »

MuffinFlavored wrote: post #165
perrygeo wrote: post #144

I'd say we have a fairly good existing theory that explains modern package management: Conway's Law. We self-organize into communities of practice and select the package management strategy that works best. Reaching across communities to develop a grand centralized strategy that fits everyone's needs would be __possible__, but involves significant communication and coordination overhead. So instead we fracture, and t…

Is there a name for a phenomena/theory that states "the more people you add to a problem while asking them to solve it in an opinionated way, the more likely you are to get conflict/fragmentation/less people agreeing overall as consensus gets diluted and number of possibilities/opinions increases?"

Maybe it should be called Lydgate’s Law: “You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time.”

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#200

Post by maximumcomfy »

A few reasons:

1) Every developer no matter how new wants to contribute something. 2) There is no centralized authority in most languages/package system to stop them doing so 3) Imagine all the bad code you write in the first 2 years but now you can't delete it because 100 other developers rely on it and 100 other developers rely on them

The rube goldberg machine exists because of the matryoshka dolls that is the dependency tree. Sometimes rewriting things is actually easier than trying to untangle the wire of dependencies.

Post reply on HN