Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disabling npm's progress bar yields a 2x npm install speed improvement

twitter.com

Post reply on HN
Print view View original

41–50 of 187 posts

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#41

Post by morebetterer »

untog wrote: post #33
morebetterer wrote: post #31

Earlier quoted context omitted.

I don't even want color and terminal graphics done goodly. I code with two monochrome terminals on the screen using vim and the command line.

Not everyone is you. While I agree that flashy graphics are silly in the command line, I find color-based highlighting to be quite useful.

> Not everyone is you.

Never claimed that was the case.

God speed with your colors. May they bring you fame and good fortune.

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#42

Post by dandelany »

krisdol wrote: post #37
yunong wrote: post #12

Earlier quoted context omitted.

Even more important, npm doesn't reliably reproduce builds. Different runs of the same shrinkwrapped build could either fail or succeed -- which is odd given that builds should be deterministic. Additionally, shrinkwrap is also broken in the same way. This is the first time I've heard of ied, a little competition could go a long way!

Could you expand on that? What causes it to be nondeterministic? I haven't had that experience but it's the second time I've read someone reference this behavior

Here are details from the docs: https://docs.npmjs.com/how-npm-works/npm3-nondet

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#44

Post by DonHopkins »

agumonkey wrote: post #30
alanh wrote: post #14

Earlier quoted context omitted.

npm 2.x user here. Does anybody want to share a video of the 3.x progress bar? (Easy with QuickTime or CloudApp, btw)

https://www.youtube.com/watch?v=elWIVKWhWTk

This seems to make it go faster:

http://bennyhillthis.com/?v=elWIVKWhWTk

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#45

Post by juandazapata »

yunong wrote: post #12
untog wrote: post #9

Progress bar or not, npm has gotten extremely slow with v3, to the extent that someone is working on a performance-centered alternative called ied: http://gugel.io/ied/ installs are literally the #1 thing npm needs to do well, and I really hope we see some improvement soon.

Even more important, npm doesn't reliably reproduce builds. Different runs of the same shrinkwrapped build could either fail or succeed -- which is odd given that builds should be deterministic. Additionally, shrinkwrap is also broken in the same way. This is the first time I've heard of ied, a little competition could go a long way!

That's frustrating, I got bitten by that a couple of times. I'm wondering when will they add a LOCKFILE like every other build system out there :'(

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#47

Post by jessaustin »

ghjnut wrote: post #32

Here's a breakdown https://github.com/npm/npm/issues/11283#issuecomment-1752468...

I guess we're supposed to assume that V8 is smart enough to inline that sort of function definition, but it seems deeply weird to me that drawBar() is defined inside the "hot" method like that. Also, I would have expected that streams would have been used for this. Logging is a very common "simple example" on node streams how-to pages.

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#48

Post by jakub_g »

libeclipse wrote: post #43

This makes sense. It's like when I'm solving a project euler problem, I don't use verbose output because it runs much faster without having to print everything to the screen. (Python)

That's exactly my experience. Once I wrote some naive PE solution in nodejs and added some logging at each step. After few minutes and far from a solution I stopped it and disabled logging. I got the solution within seconds. The diff was few orders of magnitude, which was pretty surprising for me.

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#49

Post by nijiko »

I'd also like to take the time to mention that the caching system when installing / other actions is extremely inefficient (for my times I have the progress bar turned off already and this is a project with around 80 modules shared between dev / prod):

https://github.com/npm/npm/issues/10890

I started on a very (I would like to emphasize very a thousand times over) basic proof-of-concept to show how much faster it could be in the order of magnitudes:

https://gist.github.com/nijikokun/2f1f16325f8ffe14b1b3

All this does is build a json of every package you currently have installed, and utilizes that as a lookup store the next time instead of rebuilding it every install; this was targeted towards installing / uninstalling existing packages. Not fresh installs.

Fresh installs would benefit from bulk lookups via the API imo.

Re: Disabling npm's progress bar yields a 2x npm install speed improvement

#50

Post by arjie »

jakub_g wrote: post #48
libeclipse wrote: post #43

This makes sense. It's like when I'm solving a project euler problem, I don't use verbose output because it runs much faster without having to print everything to the screen. (Python)

That's exactly my experience. Once I wrote some naive PE solution in nodejs and added some logging at each step. After few minutes and far from a solution I stopped it and disabled logging. I got the solution within seconds. The diff was few orders of magnitude, which was pretty surprising for me.

Sometimes writing to the terminal can be the slowest part because it's unbuffered by default in an interactive prompt, so even just redirecting stdout to /dev/null (or even a file) can be faster. Had that happen some ten years ago, so my explanation could be out of date or misremembered but the effect was real.
Post reply on HN