Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#161

Post by tsimionescu »

mseepgood wrote: post #110
tsimionescu wrote: post #46

Earlier quoted context omitted.

Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increas…

> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number That's simply not true. That's only one way you can do it. Another way is to create a branch.

It's the recommended way, but you're right it's not the only one. Still, updating your major version number is much harder than in any other version control system I've seen (since however you do it, it requires you to update every single file in your repo to point to the new module). It also complicates the relationship between git tags, go.mod file location, and location within the Git repo significantly.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#162

Post by Linux-Fan »

emmelaich wrote: post #19

> we apparently have no "package management theory" Honestly, I'd like to have a lowest common denominator of "just dump the contents of the tar/zip archive" here . Only check required would be to ensure you're not overwriting anything. Dependencies and PATH management etc could be done out-of-band. That way you would not have to re-download stuff at least.

I actually wrote something of that sorts for my personal projects:

https://masysma.net/11/maartifact.xhtml

Downloads tgz if not already downloaded, saves it to directory `../x-artifacts` and optionally extracts it.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#163

Post by vt85 »

anthk wrote: post #97
mschuster91 wrote: post #84

Earlier quoted context omitted.

> First, one never touches the system Python. It's there for OS-managed stuff and to run OS components. Why would one want to not use what the system provides? Everything not provided by the OS is additional maintenance burden on myself. The 'nix world has managed just fine with OS-provided bash, perl and other runtime dependencies for decades. > Then, every Python-based tool should be installed in a separate, unpriv…

Then when your system installed Python begins to crap out your whole system down, don't ask for solutions.

[dead]

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#164

Post by Cieric »

metaltyphoon wrote: post #145
Cieric wrote: post #139

Earlier quoted context omitted.

Not the one you we're replying to, but I have an idea of the arcane magic he might be referring to. Back when I started learning Go I wanted to make a few example projects to test how well the language works, but I didn't want to push anything anywhere. When I tried to break my project into modules it ended up being that I needed to give them fake urls or something and then tell it to redirect the fake url to a folde…

Now a days you can use go workspaces to do this so you don't have to modify your go.mod files anymore.

Thanks for the info, I'll look into that the next time I pick up go again.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#165

Post by MuffinFlavored »

perrygeo wrote: post #144

I'd say we have a fairly good existing theory that explains modern package management: Conway's Law. We self-organize into communities of practice and select the package management strategy that works best. Reaching across communities to develop a grand centralized strategy that fits everyone's needs would be __possible__, but involves significant communication and coordination overhead. So instead we fracture, and t…

Is there a name for a phenomena/theory that states "the more people you add to a problem while asking them to solve it in an opinionated way, the more likely you are to get conflict/fragmentation/less people agreeing overall as consensus gets diluted and number of possibilities/opinions increases?"

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#166

Post by jrockway »

You mention Python and Node, which are programming language that unusually require end users to have the text of your program and all its dependencies on their own machine, and the languages store some parts of your program in /usr/lib and other parts of your program in your source directory. (npm does a little better here and at least puts the dependencies in your project directory.) Those constraints make development and packaging hard no matter what.

Python and Node both need a way to compile the code down to a single statically-linked binary like more modern languages (Go, Rust), solving the distribution problem once and for all.

There are module systems that aren't insane, like Go's module system. It uses semantic versioning to mediate version conflicts. Programs can import multiple major versions of the same module. The module requirements files ensure that every checkout of the code gets the exact same bytes of the code's dependencies. The compiler is aware of modules and can fetch them, so on a fresh workstation, "go test ./..." or "go install ./cmd/cool-thing" in a module-aware project works without running any other command first. It is actually so pleasant to use that I always think twice "do I want to deal with modules" before using a language like Javascript or Python, and usually decide "no".

npm and pip are the DARK AGES. That's why you're struggling. The community has been unable to fix these fundamental flaws for decades, despite trying in 100 incompatible ways. I personally have given up on the languages as a result. The standard library is never enough. You HAVE to solve the module problem on day 1.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#167

Post by TillE »

jerf wrote: post #149

A solution can not be simpler than the problem. The problem is a classic "looks simple until you really examine it".

This is the answer. If you really want to learn about the issue, try designing a package manager in detail, and be sure to handle the thousands of edge cases which arise in real world usage.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#168

Post by anderskaseorg »

goodpoint wrote: post #159
tazjin wrote: post #78

Earlier quoted context omitted.

> are we thinking about Debian packaging in completely different ways? Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it woul…

> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU.

In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility.

Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#169

Post by briffle »

jrockway wrote: post #166

You mention Python and Node, which are programming language that unusually require end users to have the text of your program and all its dependencies on their own machine, and the languages store some parts of your program in /usr/lib and other parts of your program in your source directory. (npm does a little better here and at least puts the dependencies in your project directory.) Those constraints make developme…

if npm and pip are DARK AGES what does that mean perl's CPAN is?

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#170

Post by 0x0 »

anderskaseorg wrote: post #168
goodpoint wrote: post #159

Earlier quoted context omitted.

> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected…

Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
Post reply on HN