> I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally
So when you say "npm staff at the time", do you mean at the time of node 0.10?
> and knowing under which circumstances package lock files were implemented.
> Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time)
The amusing thing about your comment here is the parts which are accidentally correct.
`package-lock.json` files use the same file format as `npm-shrinkwrap.json` files. Always have, although of course the format of this file has changed significantly over the years, most dramatically with npm 7.
The "design" of the shrinkwrap/package-lock file was done rather quickly, since it was initially just a JSON dump of (most of) the data structure that npm was already using for dependency tree building. However, as far as I know, the days were the standard length of 24 hours, so while that may be "short", certainly shorter than I'd often prefer, they were (as far as I know) no shorter than any other days.
This was indeed shipped without any review by even a single "npm employee", which should not surprising, as "npm" was not at that time a legal entity capable of hiring employees. The initial work was done by Dave Pacheco, and reviewed by npm's author (at that time its sole committer and entire development staff), both of whom were Joyent employees at the time.
The use of a shrinkwrap as a non-published normal-use way to snapshot the tree at build time and produce reproducible builds across machines and time was not implemented by default until npm v5, but there wasn't really much to rush, on that particular feature. You could argue that npm 5 itself was rushed, and that's probably a fair claim, since there was some urgency to ship it along with node version 8, so as not to wait a year or more to go out with node v10.
> So this change got pushed out, had an absolute mountain of bugs that took ages to fix...
Idk, I think calling it a "mountain" is relative, actually ;)
> They're super, super broken by design.
I know you're using this phrase "broken by design" in the same sense as the author of the OP means it, but... has language just changed on me here, and I didn't notice?
As I've always heard the term used, something is "broken by design" when the actual intent is for a system to fail in some way, to achieve some goal. For example, a legislative or administrative process that is intentionally slow-moving and unable to accomplish its goals in a reasonable time frame, with the hope that this leaves room for independent innovation. Or a product that requires some minor upgrade or repair to continue working, so that the seller can keep tabs on their customers more easily. That kind of thing.
I think what you mean is not that it's "broken by design", but rather it's "a broken design". Unless this is like "begging the question", and I should just accept that I'm gradually coming to speak a language of the past, while the future moves on. It's certainly not intended to cause problems, as far as I'm aware.
If you really do mean "broken by design" (in the sense of a tail light that goes out after 50k miles so that you will visit the dealership and they can sell you more stuff), I'm super curious what you think npm gets out of it.
> Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
As of npm v7, there's no longer any practical reason why it can only audit the lockfile, rather than the actual tree on disk. Just haven't gotten around to implementing that functionality. If you want it changed, I suggest posting an issue https://github.com/npm/cli/issues. There's some question as to whether to prioritize the virtual tree or the actual tree, since prioritizing the actual tree would be a breaking change, but no reason why it can't fall back to that if there's no lockfile present.
But even approaching build reproducibility is impossible without lockfiles. If a new version of a transitive dependency is published between my install and yours, we'll get different package trees. If we both install from the same lockfile, we'll get the same package tree. (Not necessarily the same bytes on disk, since install scripts can change things, but at least we'll fetch the same bytes, or the build will fail.)
> So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
You do not recall correctly, sorry. (Or maybe you correctly recall an incorrect explanation?) The answer is reproducible builds. Using a lockfile does reduce network utilization in builds, but not very significantly.
> Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
I hope my tone is civil and playful enough in this message to not consider my response "yelling".