Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#31

Post by tazjin »

> In contrast, we apparently have no "package management theory". We have not figured out the primitives required to express dependencies

We have a good hunch. The basic theory behind Nix definitely goes in the right direction, and if we look away from all the surface-level nonsense going on in Nix, it's conceptually capable (e.g. [0]) of being a first-class language dependency manager.

For this to work at scale we'd need to overcome a couple of large problems though (in ascending order of complexity):

1. A better technical implementation of the model (working on it [1]).

2. A mindset shift to make people understand that "binary distribution" is not a goal, but a side-effect of a reasonable software addressing and caching model. Without this conceptual connection, everything is 10x harder (which is why e.g. Debian packaging is completely incomprehensible - their fundamental model is wrong).

3. A mindset shift to make people understand that their pet programming language is not actually a special snowflake. No matter what the size of your compilation units is, whether you call modules "modules", "classes" or "gorboodles", whether you allow odd features like mutually-recursive dependencies and build-time arbitrary code execution etc.: Your language fits into the same model as every other language. You don't have to NIH a package manager.

This last one is basically impossible at the current stage. Maybe somewhere down the line, if we manage to establish such a model successfully in a handful of languages and people see for themselves, but for now we have to just hold out.

[0]: https://code.tvl.fyi/about/nix/buildGo

[1]: https://cs.tvl.fyi/depot/-/tree/tvix/

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#32

Post by pastage »

Why do we need different systems for all languages, all Linux/Windows/Mac/Android distributions. Why are no one reusing what is already there. It might seem like an easy problem to fix. Rust, Java and Go certainly has their problems too, not quite sure why people feel they are good examples.

I think the world is complicated and you will in the end need to support compilations/installations on AmigaOS 2.05 on m68k running in kubernetes native.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#33

Post by tazjin »

Sebb767 wrote: post #28
denton-scratch wrote: post #18

It annoys me that every gee-whizz new language needs to have it's own package-management system. There's no reason why a package-management system needs to be language-specific; dependencies are often cross-language. Hell, even some blocks of code contain more than one language. The package-management system is responsible for deploying packages. The way a package is deployed should depend on the operating environmen…

> There's no reason why a package-management system needs to be language-specific; dependencies are often cross-language. Hell, even some blocks of code contain more than one language. This sounds a lot like a case of https://xkcd.com/927/ . Languages have different ways of importing and installing dependencies, trying to create a package manager over all of those is just going to end up making things even more compl…

> Languages have different ways of importing and installing dependencies

They're not actually different. They call things differently, and they have different methods of passing the required lookup paths/artifacts/sources to their compilers/interpreters/linkers, but in the end all of them are conceptually the same thing.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#34

Post by neilv »

The Racket module system, designed by Matthew Flatt, is great.

But to fully appreciate it, it helps to understand syntax transformation in Racket. Once the rigorous phase system forces non-kludgy static rules about when things are evaluated, your syntax transformers and the code on which they depend could cause a mess of shuffling code among multiple files to solve dependency problems... until you use submodules with the small set of visibility rules, and then suddenly your whole tricky package once again fits in a single file cleanly.

I leveraged this for some of my embedded doc and test experiments, without modifying the Racket core. (I really, really like single-source-file modules that embed doc, test, and package metadata all in the same file, in logical places.)

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#35

Post by karel_3d »

I like golang package management in practice - sure, you have the "v2" thing and "every package can bump any package if you're not careful", but it works in practice because people are careful, and the minimal version selection is always deterministic.

The proxy thing and the drama with sr.ht is annoying, but maybe it was solved while I didn't look.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#36

Post by pastage »

choeger wrote: post #8

We have such a theory, it's called "modules". But unfortunately, modules are even less popular than proper typesystems have been for decades. It's no coincidence that Rust (which has basically copied Haskell's type system to a large degree) and Julia (which is essentially a Lisp in disguise) have somewhat sound packaging systems. C/C++ does not even have a tidy core language, let alone proper modules (yes, yes, C++ m…

I do not understand what you mean by modules, and how Rust and not Java or Python is helped by this.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#37

Post by omgtehlion »

verdverm wrote: post #26
omgtehlion wrote: post #16

Earlier quoted context omitted.

Idk about Go, but Rust’s cargo seems nice, clean yet powerful. That was my impression some time ago. But last week I attempted to compile a couple of (not very big) tools from cargo. And it ended up downloading hundreds of dependencies and gigabytes of packages. Looks like node_modules.jpg all over again :(

> Idk about Go I wrote a post highlighting Go's mod system: https://verdverm.com/go-mods/ imo, it is the best designed dependency system I know of. One of the nice things is that Go uses a shared module cache so there is only one copy on your computer when multiple projects use the same dependency@version

I think, this is not a problem with package manager per se. But with extremities of coding culture. On one side of the spectrum is “NIH” and reinvent everything yourself. On the other side is: let’s pull left-pad from package, because packages are good, we need MORE PACKAGES.

The best solution always lies somewhere in the middle. But finding this “middle” (and adhering to this approach) is the hard part.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#38

Post by drewcoo »

Because of a surplus of worn out boots, candles, chickens, bowling balls (all bygone common, everyday, understandable items), and human invention (meaning "copying" badly, wheel reinvention - seldomly punctuated with novel synthesis).

Or was that open source software? Or was that any private corporate software ecosystem? If I squint the problems all start to look the same. I must need glasses, considering my squint frequency.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#39

Post by junon »

omgtehlion wrote: post #16
jakewins wrote: post #6

Have you used Go or Rust package ecosystems? My experience is that the older gen languages you mention had to invent package management, made lots of understandable mistakes and now are in a backwards compat hellscape. Rust and Go built their packaging story with the benefit of lessons learned from those other systems, and in my experience the difference is night and day.

Idk about Go, but Rust’s cargo seems nice, clean yet powerful. That was my impression some time ago. But last week I attempted to compile a couple of (not very big) tools from cargo. And it ended up downloading hundreds of dependencies and gigabytes of packages. Looks like node_modules.jpg all over again :(

As someone who contributed somewhat extensively to the node_modules problem early on, Cargo is definitely better than the JS ecosystem in this regard.

Further, another major difference is that you don't need those dependencies after you've built. You can blow them away. Doing that with node is not as straightforward, and in many cases, not possible.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#40

Post by randomdata »

denton-scratch wrote: post #18

It annoys me that every gee-whizz new language needs to have it's own package-management system. There's no reason why a package-management system needs to be language-specific; dependencies are often cross-language. Hell, even some blocks of code contain more than one language. The package-management system is responsible for deploying packages. The way a package is deployed should depend on the operating environmen…

> Git is privately-owned, and stuffed with all kinds of unaudited junk. [...] I've been using Debian derivatives for years

Git is owned by the same owner as Linux. If you've been using Debian derivatives for years it seems you must have some trust to give that private entity? Unless the derivative you speak of is Debian GNU/k*BSD?

Furthermore, if you give trust to Debian derivative projects, why not trust their Git builds? If you trust everything else in their distribution Git is a curious omission. Do you have a personal beef with Torvalds or something?

Post reply on HN