Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#141

Post by skovati »

tsimionescu wrote: post #46
jakewins wrote: post #6

Have you used Go or Rust package ecosystems? My experience is that the older gen languages you mention had to invent package management, made lots of understandable mistakes and now are in a backwards compat hellscape. Rust and Go built their packaging story with the benefit of lessons learned from those other systems, and in my experience the difference is night and day.

Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increas…

Interesting read on the official golang.org proxy DoSing git.sr.ht

https://sourcehut.org/blog/2023-01-09-gomodulemirror/

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#142

Post by michael1999 »

Part of it is deliberately ignoring the lessons of the past because they seem like overkill. Python has the excuse that it is old enough to predate the lessons of Java and Maven. But the node and js worlds deliberately ignored things like namespaces and mandatory version specification because they were uncool. And then they had to introduce package-lock and typo-squatting defences because the complexity of the world isn't always negotiable.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#143

Post by fragmede »

carlmr wrote: post #108
noobermin wrote: post #106

Earlier quoted context omitted.

I think 3 is probably true for language developers, but for users, language repos feel like they are needed because like none of the linux distros really ship everything you need, or even a reliable fraction without installing a bunch of dumb hacks to get it to work. It's much easier just to do "pip install" so that's where the demand is. And sure, may be you're right that distro packaging is the "wrong model," again…

Apart from not shipping what you need, you often only get a fraction of the versions you need. For development you may want to test your code against multiple versions of the system libraries. This is not easy using a distro package manager.

Maybe our definition of "easy" doesn't agree, but for Ksplice, we had to build kernels using the exact same, often old versions of GCC and everything else (binutils, etc). We made schroots (this predated docker) and then we used the system package manager to install the requisite old version of packages. Wasn't trivial to get the whole system setup, but getting old versions of packages installed wasn't impossibly hard either.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#144

Post by perrygeo »

I'd say we have a fairly good existing theory that explains modern package management: Conway's Law.

We self-organize into communities of practice and select the package management strategy that works best. Reaching across communities to develop a grand centralized strategy that fits everyone's needs would be __possible__, but involves significant communication and coordination overhead. So instead we fracture, and the tooling ecosystem reflects ad-hoc organizational units within the community.

Ecosystems like Rust cargo that have batteries included from the start have an advantage, virtually all Rust developers have a single obvious path to package management because of this emergent social organization.

Ecosystems like Python's seem like the wild west, there is deep fracturing (particularly between data science and software engineering) and no consensus on the requirements or even the problems. So Python fractures further, ironically in a search for something that can eventually unify the community. And python users feel the strain of this additional overhead every day, needing to side with a team just to get work done.

I'd argue both of these cases are driven by consequences easily predictable from Conway's Law.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#145

Post by metaltyphoon »

Cieric wrote: post #139
jen20 wrote: post #105

Earlier quoted context omitted.

Some of your criticism is reasonable, and I’m no fan of Go’s module system as a standalone artifact, but much of your criticism is unfounded. > It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. This is untrue. > It requires users of your code to update all of their import statements throughout their code whenever you move your hosting…

Not the one you we're replying to, but I have an idea of the arcane magic he might be referring to. Back when I started learning Go I wanted to make a few example projects to test how well the language works, but I didn't want to push anything anywhere. When I tried to break my project into modules it ended up being that I needed to give them fake urls or something and then tell it to redirect the fake url to a folde…

Now a days you can use go workspaces to do this so you don't have to modify your go.mod files anymore.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#146

Post by thenerdhead »

Have you read https://medium.com/@sdboyer/so-you-want-to-write-a-package-m... ?

The author does a great job pointing out the problems, theory, and ecosystem change that makes it a Rube Goldberg machine.

By definition, a package manager is always "incomplete" because it cannot catch all the security vulnerabilities, binary compatibility, or knowing the dependencies ahead of time nor their interactions together. Thus it can be unsuccessful at managing dependencies - its primary job.

Source: Also work on a notable package manager.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#147

Post by jmull »

qbasic_forever wrote: post #117

There is no real theory because it's a solved problem--packages and dependencies form a DAG or graph and it's just an ordering and traversal problem that's easily solved. The pain and fragmentation you're mentioning is that everyone has different opinions about how they want to configure/bundle/organize code and package metadata. That's really the only core difference between deb/rpm, npm/yarn, pip/poetry/conda/ , ha…

Well, not DAG, because the A is for acyclic, and dependency graphs can definitely contain cycles.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#148

Post by tomcam »

pjmlp wrote: post #90
tomcam wrote: post #79

Earlier quoted context omitted.

Can you give me a one-liner about module drama in Go?

First they weren't supported, so the community created various ways of dealing it including a kind of Google's blessed implementation, then Google decided to create their own official way, then there was the transition time which I guess not everyone has done, having SCM urls as imports is just bad, and how GOPROXY DoS some SCM repos outside of Github.

Thank you. I didn’t quite see all that happening but heard echoes of it from time to time, then really jumped in after they adopted modules.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#150

Post by tomcam »

Nzen wrote: post #125
pjmlp wrote: post #90

Earlier quoted context omitted.

First they weren't supported, so the community created various ways of dealing it including a kind of Google's blessed implementation, then Google decided to create their own official way, then there was the transition time which I guess not everyone has done, having SCM urls as imports is just bad, and how GOPROXY DoS some SCM repos outside of Github.

[0] https://go.dev/blog/appengine-gopath gopath / workspaces original [1] https://go.dev/blog/migrating-to-go-modules current system [2] https://news.ycombinator.com/item?id=34310674 goproxy agressively polling sourcehut It's worth recognizing that go isn't alone in dns-based namespacing : java's maven/gradle use the same strategy: https://repo1.maven.org/maven2/gov/nih/imagej/imagej/1.47/

Fantastic links, thanks! I like the DNS-based way so much I am still trying to tease apart its disadvantages.
Post reply on HN