Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#121

Post by DemocracyFTW2 »

Izkata wrote: post #111
DemocracyFTW2 wrote: post #57

Earlier quoted context omitted.

> I don't think anyone has significant issues with pip and boy would you be wrong. I just tried to install `datasette-scraper` on my system. This didn't work because it would consistently pick up outdated versions of Django and `more-itertools` from the system's Python site-packages directory. By the way I learned to (1) install an updated version of Python that has the advantage of not being the one that the system…

> no 'virtual-env' or some such! I'm pretty sure you're manually doing the same thing virtualenv (venv in python3) does for you...

I'd guess that, yes, and going forward I may take a 2nd look at virtualenv. That said, I am a little proud and glad I managed to take it this far in this particular instance because I feel it has improved my ability to deal with Pythonic installation gotchas. Pip is a great teaching tool! /s

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#122

Post by jakewins »

gwd wrote: post #76
tsimionescu wrote: post #46

Earlier quoted context omitted.

Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increas…

> It relies on DNS for package identity. The flip side of this is that it never has to worry about naming collisions or namespacing: Your public package name must be a URL you control. Additionally, there is no requirement for a centralized package facility to be run. The Golang project is currently running pkg.go.dev, but that's only been in the last few years; and if they decided to get rid of it, it wouldn't signi…

Java enforces domain name ownership as well; I can’t publish to maven central under io.netty.

This removes a whole slew of attack surfaces that JS and Python have, and leverages an existing “prove you are who you say you are” infra

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#123

Post by dmix »

mijoharas wrote: post #93
folex wrote: post #44

Earlier quoted context omitted.

Yes! I'm in a team that works on a pet prog lang for distributed systems, and we did some research of using an existing package managing systems. We've settled on NPM for now, but god I wish there would be a better generic package manager out there.

Not a generic package manager, but it's probably worth calling out asdf as the generic version manager[0] (maybe you're already aware of it, but it's a generic replacement for nvm, rvm, virtualenv, *vm, which supports any language based on plugins.) Again, maybe you're already aware of it, but I think it's a nice example of genericising a concern common to many languages which sounds similar to what you're asking for…

At a minimum it's useful having the `.tool-versions` dotfile in a project directory which lists the versions of each language, db, and package manager version (our Rails project lists versions for ruby, bundler, postgres, node, yarn, redis). Even if all devs don't use asdf, it's a useful reference point.

https://asdf-vm.com/manage/configuration.html#tool-versions

Similar to an `.editorconfig` to globally define stuff like tab widths for IDEs.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#124

Post by puffoflogic »

tazjin wrote: post #31

> In contrast, we apparently have no "package management theory". We have not figured out the primitives required to express dependencies We have a good hunch. The basic theory behind Nix definitely goes in the right direction, and if we look away from all the surface-level nonsense going on in Nix, it's conceptually capable (e.g. [0]) of being a first-class language dependency manager. For this to work at scale we'd…

Nix is not capable of becoming a first class dependency manager because it does not manage dependencies. It does not attempt any version or feature resolution. Besides, saying Nix could become a package manager is like saying that C could become a package manager.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#125

Post by Nzen »

pjmlp wrote: post #90
tomcam wrote: post #79

Earlier quoted context omitted.

Can you give me a one-liner about module drama in Go?

First they weren't supported, so the community created various ways of dealing it including a kind of Google's blessed implementation, then Google decided to create their own official way, then there was the transition time which I guess not everyone has done, having SCM urls as imports is just bad, and how GOPROXY DoS some SCM repos outside of Github.

[0] https://go.dev/blog/appengine-gopath gopath / workspaces original

[1] https://go.dev/blog/migrating-to-go-modules current system

[2] https://news.ycombinator.com/item?id=34310674 goproxy agressively polling sourcehut

It's worth recognizing that go isn't alone in dns-based namespacing : java's maven/gradle use the same strategy: https://repo1.maven.org/maven2/gov/nih/imagej/imagej/1.47/

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#127

Post by namelosw »

Modern languages like Rust and Elixir are fine.

JavaScript, C/C++, Python are very old and the package systems were pretty much bolt-on. And of course a lot of historical factors were playing a bit role.

In the old days, people usually start minimal. When they get burned they also find they were locked-in - thus the bolt-ons.

> we apparently have no "package management theory"

People are definitely standing on top of each others' shoulders. Node.js npm actually felt like Ruby package tools but also being first-party (they also cut some corners, and made some design choices - some worked well and some didn't).

i.e. Hex for Elixir feels like nothing new, but avoided most common pitfalls others have experienced. I believe most language designers nowadays can do the same, given they want to play it safe and want nothing too novel.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#128

Post by Groxx »

verdverm wrote: post #26
omgtehlion wrote: post #16

Earlier quoted context omitted.

Idk about Go, but Rust’s cargo seems nice, clean yet powerful. That was my impression some time ago. But last week I attempted to compile a couple of (not very big) tools from cargo. And it ended up downloading hundreds of dependencies and gigabytes of packages. Looks like node_modules.jpg all over again :(

> Idk about Go I wrote a post highlighting Go's mod system: https://verdverm.com/go-mods/ imo, it is the best designed dependency system I know of. One of the nice things is that Go uses a shared module cache so there is only one copy on your computer when multiple projects use the same dependency@version

I really like a lot of the distribution- and safety-related decisions Go modules made. Domain names and the proxy+sumdb are wonderfully clear, flexible, and scalable, and I think we'll see copies of it in many future languages.

The rest of the stuff around modules, like crippled constraints, zero control over contents (which they have changed!), and completely non-existent "x is available, upgrade" or release tooling: constant, unnecessary pain, and it'll be inflicting serious damage on the ecosystem for many years to come.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#129

Post by thiht »

gwd wrote: post #64
thiht wrote: post #50

Earlier quoted context omitted.

Package management in Go is weird but it works well. You just run go mod tidy from time to time and you're good. Add a module proxy if you want to prevent issues with repositories disappearing and you're gooder.

There's already a module proxy (pkg.go.dev) by default in new versions of Go, for just that purpose. In fact, one could argue that the automatic proxy is "Rube-Goldberg"-like: It works great normally, but if you need, for instance, to pull from a repo you just pushed to, you have to track down the magic rune to type to get `go get -u` to pull directly from your repo, rather than using the cached copy at pkg.go.dev.

> There's already a module proxy (pkg.go.dev) by default in new versions of Go, for just that purpose

Is it really a proxy in this sense? I was under the impression that it did not keep cached copies of the code, just some metadata, so that it didn't actually protect about someone removing their repo. Maybe I'm wrong about that though.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#130

Post by derefr »

What I really want to know, is why package development is such a Rube Goldberg machine. Not for programming-language packages, per se, but rather for OS packages of simple programming-language packages.

Have you tried to package a random Python/Ruby/etc. CLI program, for Debian? Or how about for Homebrew? Each one involves a cacophony of PL scripts calling shell-scripts calling PL scripts, and internal/undocumented packager subcommands calling other internal/undocumented packager subcommands. It takes ~forever to do a checked build of a package in these systems, and 99% of it is just because of how spread out the implementation of such checks is over 100 different components implemented at different times by different people. It could all be vastly simplified by reimplementing all the checks and transforms in a single pass that gradually builds up an in-memory state, making assertions about it and transforming it as it goes, and then emitting it if everything works out. You know — like a compiler.

Post reply on HN