Live data from Hacker News

My failed attempt to shrink all NPM packages by 5%

evanhahn.com

111–120 of 253 posts

Re: My failed attempt to shrink all NPM packages by 5%

#111

My experiment on how to reduce javascript size of every web app by 30-50% : https://github.com/avodonosov/pocl Working approach, but in the end I abandoned the project - I doubt people care about such js size savings.

I got measurable decreases in deployment time by shrinking the node_modules directory in our docker images.

I think people forget that, when you’re copying the same images to dozens and dozens of boxes, any improvement starts to add up to real numbers.

Re: My failed attempt to shrink all NPM packages by 5%

#114
post #48

Earlier quoted context omitted.

> This is a solid example of how things change at scale. 5% is 5% at any scale.

5% of newly published packages, with a potentially serious degradation to package publish times for those who have to do that step. Given his numbers, let's say he saves 100Tb of bandwidth over a year. At AWS egress pricing... that's $5,000 total saved. And arguably - NPM is getting at least some of that savings by adding CPU costs to publishers at package time. Feels like... not enough to warrant a risky ecosystem c…

How often are individuals publishing to NPM? Once a day at most, more typically once a week or month? A few dozen seconds of one person's day every month isn't a terrible trade-off.

Even that's addressable though if there's motivation, since something like transcoding server side during publication just for popular packages would probably get 80% of the benefit with no client-side increase in publication time.

Re: My failed attempt to shrink all NPM packages by 5%

#116
post #8

Earlier quoted context omitted.

I felt the same. The proposal wasn't rejected! Also, performance gains go beyond user stories - e.g. they reduce infra costs and environmental impact - so I think the main concerns of the maintainers could have been addressed.

> The proposal wasn't rejected! They soft-rejected by requiring more validation than was reasonable. I see this all the time. "But did you consider ? Please go and run more tests." It's pretty clear that the people making the decision didn't actually care about the bandwidth savings, otherwise they would have put the work in themselves to do this, e.g. by requiring Zopfli for popular packages. I doubt Microsoft cares…

> which according to my brief checks costs around $10k/week

That's the market price though, for Microsoft its a tiny fraction of that.

Re: My failed attempt to shrink all NPM packages by 5%

#117
From the RFC on github[1].

> Zopfli is written in C, which presents challenges. Unless it was added to Node core, the CLI would need to (1) rewrite Zopfli in JS, possibly impacting performance (2) rely on a native module, impacting reliability (3) rely on a WebAssembly module. All of these options add complexity.

Wow! Who's going to tell them that V8 is written in C++? :)

[1]: https://github.com/npm/rfcs/pull/595

Re: My failed attempt to shrink all NPM packages by 5%

#118

Earlier quoted context omitted.

That’s an argument against making any change to the packaging system ever. “It might break something somewhere” isn’t an argument, it’s a paralysis against change. Improving the edge locality of delivery of npm packages could speed up npm installs. But speeding up npm installs might cause the CI system which is reliant on it for concurrency issues to have a race condition. Does that mean that npm can’t ever make it f…

It is an argument. An age old argument: "If it ain't broke, don't fix it."

disable PRs if this is your policy.

Re: My failed attempt to shrink all NPM packages by 5%

#119
post #3

The final pro/cons list: https://github.com/npm/rfcs/pull/595#issuecomment-1200480148 I don't find the cons all that compelling to be honest, or at least I think they warrant further discussion to see if there are workarounds (e.g. a choice of compression scheme for a library like typescript, if they would prefer faster publishes). It would have been interesting to see what eventually played out if the author hadn't…

The pros aren't all that compelling either. The npm repo is the only group that this would really be remotely significant for, and there seemed to be no interest. So it doesn't take much of a con to nix a solution to a non-problem.

Re: My failed attempt to shrink all NPM packages by 5%

#120
post #48

Earlier quoted context omitted.

> This is a solid example of how things change at scale. 5% is 5% at any scale.

5% of newly published packages, with a potentially serious degradation to package publish times for those who have to do that step. Given his numbers, let's say he saves 100Tb of bandwidth over a year. At AWS egress pricing... that's $5,000 total saved. And arguably - NPM is getting at least some of that savings by adding CPU costs to publishers at package time. Feels like... not enough to warrant a risky ecosystem c…

https://www.reddit.com/r/webdev/comments/1ff3ps5/these_5000_...

NPM uses at least 5 petabytes per week. 5% of that is 250 terabytes.

So $15,000 a week, or $780,000 a year in savings could’ve been gained.

Post reply on HN