Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does every package+module system become a Rube Goldberg machine?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 222 posts

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#131

Post by henrydark »

staunton wrote: post #22
Hendrikto wrote: post #13

Earlier quoted context omitted.

Tying package management to GitHub seems convenient in the short term, but will be the baggage of the next generation. I cringe hard when I see projects depending on git repos, without pinning a version or commit.

It's not actually tied to GitHub. If GitHub died tomorrow, they would easily be able to move on and host the packages somewhere else. There's no way to do this without hosting. Also, there is a way to pin a version or commit. Julia for example always stores the exact commit information for all packages in the "Manifest" file. There are also straightforward ways to demand certain versions and package maintainers have…

Strictly speaking, hosting like github isn't the only way, an alternative being some peer to peer network like torrents

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#132

Post by megous »

derefr wrote: post #130

What I really want to know, is why package development is such a Rube Goldberg machine. Not for programming-language packages, per se, but rather for OS packages of simple programming-language packages. Have you tried to package a random Python/Ruby/etc. CLI program, for Debian? Or how about for Homebrew? Each one involves a cacophony of PL scripts calling shell-scripts calling PL scripts, and internal/undocumented p…

That's a Debian specific quirk. There are simpler, more cohesive package management systems, like eg. pacman, where package maintainers's tools mastery is not an arcane art in itself.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#133

Post by rkrzr »

potamic wrote: post #59
tazjin wrote: post #31

> In contrast, we apparently have no "package management theory". We have not figured out the primitives required to express dependencies We have a good hunch. The basic theory behind Nix definitely goes in the right direction, and if we look away from all the surface-level nonsense going on in Nix, it's conceptually capable (e.g. [0]) of being a first-class language dependency manager. For this to work at scale we'd…

What's a good source, for someone who has never used nix, to read about its conceptual model and how it is different from contemporary dependency managers?

We wrote a blog post on how Nix helps us solve some of the problems that we encountered with other package managers like e.g. apt and pip.

https://www.channable.com/tech/nix-is-the-ultimate-devops-to...

(It doesn't go very much in depth on the conceptual model, but touches on the the main ideas)

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#134

Post by tazjin »

puffoflogic wrote: post #124
tazjin wrote: post #31

> In contrast, we apparently have no "package management theory". We have not figured out the primitives required to express dependencies We have a good hunch. The basic theory behind Nix definitely goes in the right direction, and if we look away from all the surface-level nonsense going on in Nix, it's conceptually capable (e.g. [0]) of being a first-class language dependency manager. For this to work at scale we'd…

Nix is not capable of becoming a first class dependency manager because it does not manage dependencies. It does not attempt any version or feature resolution. Besides, saying Nix could become a package manager is like saying that C could become a package manager.

[flagged]

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#135

Post by EFreethought »

simiones wrote: post #49
conor_f wrote: post #5

At least in Python's case, I think people are much too hard on the packaging system. I don't think anyone has significant issues with pip itself as opposed to the runtime it exists within. Following from this, I think that most back-end applications should try solve their messy runtime environment issues with some containerization. Java/C(++) however...

Java's Maven has been the nicest package management tool I've ever worked with. Apart from the fact that definitions are a bit verbose, it supported everything I've ever wanted from such a tool: 1. Clear definitions of what is an artifact that it manages, easily described in a versioned file but still separate from your source code 2. Simple support to combine internal and external sources 3. Support for interacting…

I would like to second that Maven works for me as well.

A lot of posters have said that Rust and Go work well.

Honestly, the OP sounds like the stereotype of a JS developer: "This is a problem for Javascript, therefore it must be a problem for everyone else, too."

No, guys, it's just you. Every single time.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#136

Post by deltarholamda »

namelosw wrote: post #127

Modern languages like Rust and Elixir are fine. JavaScript, C/C++, Python are very old and the package systems were pretty much bolt-on. And of course a lot of historical factors were playing a bit role. In the old days, people usually start minimal. When they get burned they also find they were locked-in - thus the bolt-ons. > we apparently have no "package management theory" People are definitely standing on top of…

Avoiding past pitfalls is the best part of a new language ecosystem. I remember when Python was the new kid on the block and how joyous it was to work with in comparison to Perl. By the time Ruby came around, the same thing was being said about it, sometimes with aspersions cast at Python and how wonky it was.

Maybe after a few more generational cycles we'll develop the One True Language, bug-free, intuitive tooling, and sane to use.

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#137

Post by verdverm »

Groxx wrote: post #128
verdverm wrote: post #26

Earlier quoted context omitted.

> Idk about Go I wrote a post highlighting Go's mod system: https://verdverm.com/go-mods/ imo, it is the best designed dependency system I know of. One of the nice things is that Go uses a shared module cache so there is only one copy on your computer when multiple projects use the same dependency@version

I really like a lot of the distribution- and safety-related decisions Go modules made. Domain names and the proxy+sumdb are wonderfully clear, flexible, and scalable, and I think we'll see copies of it in many future languages. The rest of the stuff around modules, like crippled constraints, zero control over contents (which they have changed!), and completely non-existent "x is available, upgrade" or release tooling…

> crippled constraints

Do you mean ranges on dep versions? The way it is currently, the version you set is the minimum, and the algo finds the highest minimum set across all deps and uses that.

If ranges were introduced, you'd end up with an NP hard problem and need a SAT solver for your deps again

> Release tooling

What are you looking for here? Libraries only need to push a git tag, binaries do require a bit of work, but Goreleaser fills that pretty nicely. It would seem hard to standard where binaries would be pushed

> completely non-existent "x is available, upgrade"

https://go.dev/doc/modules/managing-dependencies#discovering...

"go list -m -u all"

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#138

Post by marcus_holmes »

omgtehlion wrote: post #37
verdverm wrote: post #26

Earlier quoted context omitted.

> Idk about Go I wrote a post highlighting Go's mod system: https://verdverm.com/go-mods/ imo, it is the best designed dependency system I know of. One of the nice things is that Go uses a shared module cache so there is only one copy on your computer when multiple projects use the same dependency@version

I think, this is not a problem with package manager per se. But with extremities of coding culture. On one side of the spectrum is “NIH” and reinvent everything yourself. On the other side is: let’s pull left-pad from package, because packages are good, we need MORE PACKAGES. The best solution always lies somewhere in the middle. But finding this “middle” (and adhering to this approach) is the hard part.

I think I'm on the NIH side of things. Not because I like inventing things (well, partially), but because of the security problem [0].

I don't see it being talked about here, either. Our current system of pulling in packages made by random people on the internet is going to burn us. We assume that everyone who creates a package is an honest, reliable, developer who will not inject malicious code into their package. This assumption is similar to the assumptions we made around SMTP, HTTP, DNS, and every other internet protocol. Turns out we were wrong and surprise! you can't trust people on the internet.

I'm not sure if we can solve this with package managers. But package managers are part of the culture that has created this problem, and are probably a reasonable starting point to try and address it.

[0]:https://david-gilbertson.medium.com/im-harvesting-credit-car...

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#139

Post by Cieric »

jen20 wrote: post #105
tsimionescu wrote: post #46

Earlier quoted context omitted.

Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increas…

Some of your criticism is reasonable, and I’m no fan of Go’s module system as a standalone artifact, but much of your criticism is unfounded. > It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. This is untrue. > It requires users of your code to update all of their import statements throughout their code whenever you move your hosting…

Not the one you we're replying to, but I have an idea of the arcane magic he might be referring to. Back when I started learning Go I wanted to make a few example projects to test how well the language works, but I didn't want to push anything anywhere. When I tried to break my project into modules it ended up being that I needed to give them fake urls or something and then tell it to redirect the fake url to a folder instead. This[1] and this[2] gives a good example of the stuff I was running into at the time. I also realize that I should have probably been using sub-packages and not sub-modules, but coming from other languages I didn't realize they were suppose to be different. The thing that screwed with me most though was that there were half solutions for sub-modules to work so I kept using that terminology when searching google.

[1] https://stackoverflow.com/questions/10687627/relative-import... [2] https://stackoverflow.com/questions/57806081/how-to-access-n...

Re: Ask HN: Why does every package+module system become a Rube Goldberg machine?

#140

Post by rklaehn »

I find cargo (rust package manager) relatively pleasant to use. It's not perfect, but it also is not a constant source of confusion like sbt (scala package manger) was back when I did scala.

In the long term, I think the whole notion of a "package" is obsolete. In unison https://www.unison-lang.org/ , code is just a content-addressed merkle tree. So dependencies are both more precise and more fine grained than packages.

E.g. if there is a new version of a library, but the part of the library that you actually use is unchanged, unison will detect this and not trigger a full rebuild.

The whole notion of a package becomes less important.

Post reply on HN