Dependencies and resilience • Hacker News Board

Re: Dependencies and resilience

#11

Post by WorldMaker » Fri, Apr 12, 2024, 6:36 PM UTC

abathur wrote: post #7

remram wrote: post #4

Earlier quoted context omitted.

Then the code for library B still exists, still potentially has bugs, the only difference is that the same bug has to be fixed by project A1 then again by project A2 and project A3 etc. There is a cost there too, outlined in the recent article 'Tech Debt: My Rust Library Is Now a CDO' https://news.ycombinator.com/item?id=39827645

I guess there's a hybrid model where you're able to select exactly what you're depending on and pull it in dynamically at build/package time. I've thought a little about, for example, building something that could slice just the needed utility functions out of a Shell utility library. (Not really for minimizing the dependency graph--just for reducing the source-time overhead of parsing a large utility library that yo…

Isn't that already just the role of tree-shaking optimizers? At that point the problem seems to be languages that don't have good tree-shakers, don't/can't tree-shake library dependencies, or maybe that tree-shaking should happen earlier and more often than it often does?

Observably, it seems like the "granularity pendulum" in the JS ecosystem very directly related to the module system. CommonJS was tough to tree-shake so you had sometimes wild levels of granularity where even individual functions might be their own package in the dependency graph. ESM is a lot easier to tree-shake and you start to see more of the libraries that once published dozens or hundreds of sub-packages start to repackage back to just one top-level package alongside ESM adoption.

Re: Dependencies and resilience

#12

Post by abathur » Fri, Apr 12, 2024, 7:50 PM UTC

WorldMaker wrote: post #11

abathur wrote: post #7

Earlier quoted context omitted.

I guess there's a hybrid model where you're able to select exactly what you're depending on and pull it in dynamically at build/package time. I've thought a little about, for example, building something that could slice just the needed utility functions out of a Shell utility library. (Not really for minimizing the dependency graph--just for reducing the source-time overhead of parsing a large utility library that yo…

Isn't that already just the role of tree-shaking optimizers? At that point the problem seems to be languages that don't have good tree-shakers, don't/can't tree-shake library dependencies, or maybe that tree-shaking should happen earlier and more often than it often does? Observably, it seems like the "granularity pendulum" in the JS ecosystem very directly related to the module system. CommonJS was tough to tree-sha…

I imagine the answer's ~yes from the perspective of something you build and deploy (and I agree it's relevant to the article--but I'll caveat that I read xamuel to be asking the question very broadly).

Relying on a post-build process to avoid deploying unused code and dependencies still exposes you to a subset of problems with most if not all of the dependency graph.

Sufficiently-rich correct-by-definition metadata on the internal and external dependencies of each package might let you prune some branches without requiring the dependency to be present, but in the broad there are a lot of cases where that can't really help?

Re: Dependencies and resilience

#13

Post by abathur » Sat, Apr 13, 2024, 3:03 PM UTC

WorldMaker wrote: post #11

abathur wrote: post #7

Earlier quoted context omitted.

I guess there's a hybrid model where you're able to select exactly what you're depending on and pull it in dynamically at build/package time. I've thought a little about, for example, building something that could slice just the needed utility functions out of a Shell utility library. (Not really for minimizing the dependency graph--just for reducing the source-time overhead of parsing a large utility library that yo…

Isn't that already just the role of tree-shaking optimizers? At that point the problem seems to be languages that don't have good tree-shakers, don't/can't tree-shake library dependencies, or maybe that tree-shaking should happen earlier and more often than it often does? Observably, it seems like the "granularity pendulum" in the JS ecosystem very directly related to the module system. CommonJS was tough to tree-sha…

Perhaps by coincidence, there's a good post on treeshaking+wasm on the front page this morning: https://news.ycombinator.com/item?id=40023319