Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

81–90 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#81

> See JavaScript/TypeScript, Python, or C/C++. I feel like you've picked the 3 worst ecosystems as an example. Java has a wonderful ecosystem. You can use Maven/Gradle/Bazel as a dependency manager/build tool. All support the Maven repository format. Packages are published as jars and easily consumed. Of course you can still end up in dependency hell and you should take steps to mitigate those issues, but I'm quite h…

> JavaScript/Python developers are happy (or willing) to try something new, so new alternatives are created.

FWIW as far as JavaScript on NodeJS goes, npm did set the standard quite firmly. You can also use yarn or pnpm, which promise improvements over npm while using the same package metadata (`package.json`) and produce npm-compatible structures in the same subdirectory (`node_modules`), so overall the user experience is muche like going from `sudo apt-get install` to the newer, more polished and otherwise largely equivalent `sudo apt install`.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#83
If I wanted to point out what the greatest mistake in the conception of the NodeJS package manager—npm—is, its 'original sin', if you will, is the conceit that SemVer ('semantic versioning', a great idea in itself) is 100% reliable and that, therefore, it would be heretic or contradictory for a single module or sub-package in a project to ever want to `require()` two distinct versions of a module, say, xyz@1.0.1 and xyz@1.2.0. This is now fixed[1] but the developers stubbornly refused to see the occasional necessity of doing just this for many years.

[1](https://medium.com/weekly-webtips/how-to-install-multiple-ve...)

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#84
post #58

Earlier quoted context omitted.

> I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. I'm running macOS with Macports, and oh boy. For a long time, the aws and eb CLI tools had different package requirements or whatnot, and you couldn't install both of them simultaneously for whatever reason. Some stuff installs fine with the system Python, some stuff needs Macports for a specific Python version,…

System Python? Root access? Oh. That's a terrible, terrible practice. First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin). If you find yourself ever issuing a `sudo pip install`, chances…

> First, one never touches the system Python. It's there for OS-managed stuff and to run OS components.

Why would one want to not use what the system provides? Everything not provided by the OS is additional maintenance burden on myself. The 'nix world has managed just fine with OS-provided bash, perl and other runtime dependencies for decades.

> Then, every Python-based tool should be installed in a separate, unprivileged virtualenv, with executables symlinked where you find them appropriate (usually ~/bin or /usr/local/bin).

No one has time for all that. I'm not a Python developer - as a user I'm happy enough if it barely works.

When an ecosystem requires messing around with completely separate instances of the runtime for each program, that's not a good sign for the quality of the ecosystem as a whole.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#85
post #76

Earlier quoted context omitted.

Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increas…

> It relies on DNS for package identity. The flip side of this is that it never has to worry about naming collisions or namespacing: Your public package name must be a URL you control. Additionally, there is no requirement for a centralized package facility to be run. The Golang project is currently running pkg.go.dev, but that's only been in the last few years; and if they decided to get rid of it, it wouldn't signi…

People lose DNS names by accident all the time. It's also easy to typosquat many DNS domains, and even Github projects occasionally.

Non-DNS-based packages don't have to be named "go-sqlite". You can easily require some namespacing, and even use DNS as a base for that, but having an abstraction over it that recognizes the specific needs of package management is better. For example, Maven packages are called things like org.apache.commons, and registering a new package requires control of an equivalent DNS domain. However, if you later lose control of that domain, the new owners don't simply get to replace the packet in Maven just because they snipes your domain.

Go's choice to require full paths for import in each file is also not a direct implication of the previous item - they could have allowed the go.mod file to specify the path and a name, and then allow source files to import based on that name. Instead, this comes from Go tooling that existed before modules support, when the tooling would scour all files in your project to find dependencies.

Moving code to a v2 dir does not specifically help (or hinder) with backwards compatibility. Old code can always simply keep using the old versions of the package anyway. It is also a very unpopular decision, with very few packages actually adopting v2 versions precisely because of this requirement, even when making major breaking changes. Even the team maintaing the Go protobuf bindings decided not to use v2 when they rehauled their code (opting instead to create a new v1 at a new location with minor versions starting at 1.20...).

Sure, packaging is hard, but the Go team has chosen to go against the flow of essentially all other package managers, and instead of learning from their mistakes, they seem to have decided to make original mistakes all their own.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#86
post #67
post #59

Earlier quoted context omitted.

What's a good source, for someone who has never used nix, to read about its conceptual model and how it is different from contemporary dependency managers?

There really isn't one that I know of. There's the original thesis[0], but it's from a different time and it focuses only on the package management (~ for a distribution) aspect. Everything else is mostly written to teach people how to use Nix, and the more recent the thing is the more it will focus on surface-level features of the C++ implementation of Nix. [0]: https://edolstra.github.io/pubs/phd-thesis.pdf

[deleted]

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#87

It annoys me that every gee-whizz new language needs to have it's own package-management system. There's no reason why a package-management system needs to be language-specific; dependencies are often cross-language. Hell, even some blocks of code contain more than one language. The package-management system is responsible for deploying packages. The way a package is deployed should depend on the operating environmen…

Probably because your Debian doesn't work on other Linux distros like CentOS, Arch Linux ..., let alone MacOS and Windows. The holy grail would be a package manager that is both cross language and cross platform at the same time, but it seems too hard to be implemented in general.

Anaconda is probably the closest as it also package many non python packages. Nix is also similar but it doesn't support Windows at all (without WSL)

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#88
I am starting to see some consistency these days. I mostly work in Go, so I'm not intimately familiar with the state of the art in other languages, but it looks like most ecosystems have a tool that support this flow.

A declarative (go.mod) file that always installs the same dependencies every build, a tool to update that lock file (go mod tidy), and an indication in the source that indicates how the dependency is being used (import).

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#89
post #24

I think package management is something so important you either have to get it right on the first released version of the language, or never release an official one at all. So if you look at C, there was basically no internet or even HTTP, so if they wanted to fetch + store dependencies from centralized repositories, it would have been difficult and then need to change a lot over the years. Python is the worst to me,…

I’ve also found it not that much harder to do scripting kinds of things in Rust vs Ruby or Python. It’s a bit of extra effort of course but having a nice self contained binary at the end is pretty handy.

We've been using Go to build CLI tools rather than writing scripts for certain tasks recently, and I've found that Golang is very well suited for that.

we use the urfave/cli library, which works great, and gets you a ton of things for almost free (passing and documenting flags and arguments) that you just don't get easily in bash

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#90
post #79
post #20

Earlier quoted context omitted.

Doesn't look like it, see module drama in Go, while Rust is having an npm like ecosystem of tiny crates. Plus none of them handle binary library distribution as some of the packing models that came before them.

Can you give me a one-liner about module drama in Go?

First they weren't supported, so the community created various ways of dealing it including a kind of Google's blessed implementation, then Google decided to create their own official way, then there was the transition time which I guess not everyone has done, having SCM urls as imports is just bad, and how GOPROXY DoS some SCM repos outside of Github.
Post reply on HN