Hi folks, npm COO here. This was an operational issue that we worked to correct. All packages are now restored: https://status.npmjs.org/incidents/41zfb8qpvrdj
Isaac is subhuman filth.
Many packages suddenly disappeared
321–330 of 520 posts
Re: Many packages suddenly disappeared
#322Yikes, what is it about node/npm/javascript that makes it feel like a house of cards?
Yikes, what is it about node/npm/javascript that makes it feel like a house of cards? I think the (short) answer is "node, npm, and javascript". The longer answer has something to do with the automatic installation of dependencies, and the common use of shell scripts downloaded directly off the internet and executed using the developer's or sysadmin's user account. I used to use CPAN all the time. CPAN would check de…
I had very limited contact with CPAN some years ago but I imagine it was slightly more sane in terms of granularity of dependencies.
Whereas a lot of npm modules are relatively small - some tiny - and have their own dependencies. So a simple "npm install blah" command can result in dozens of packages being installed. Dealing with that manually would, in fairness, be a giant chore.
Now of course there's a discussion to be had about whether thousands of weeny little modules is a good idea or not but, to be honest, that's a religious debate I'd rather steer clear of.
Re: Many packages suddenly disappeared
#323I just don't understand how this can happen. In Maven Central for example (Java) if you publish a package it is immutable and stays there until nuclear fire immolates the Earth.
Re: Many packages suddenly disappeared
#324Earlier quoted context omitted.
At my job we do native C and C++, some Java, some C#, scripting in Shell, Python, and Perl. When the left-pad incident happened someone said something to the room about it, we all looked it up, and spent a good 15 minutes mind-boggled, laughing and being grateful we weren't web devs. "Wait, you're telling me these people need NPM and GitHub to deploy? Seriously?"
>these people need [their package manager] and [their source code management tool] to deploy? Seriously? Not really sure I understand what you're implying there
I'm not the poster you're replying to, but I think I understand it.
npm is not just their package management tool... the way most people use it, it depends on someone else's package registry/repository to deploy to your own servers.
And github is someone else's source code management tool/server.
As a matter of policy, if I can't have something on my own server (or one my org controls) I don't get to rely on it to deploy/run my application.
So I think I get the parent's comment... it's a really foreign situation, to me, to depend on the availability of stuff like this on servers I (or my org) don't control in order to deploy my application.
I'm sure the people who depend on these things look at me and say "Wait. You have to set up your own package repository and source control before you can deploy instead of using all this nice stuff that's available in the cloud? Seriously?"
Re: Many packages suddenly disappeared
#325Earlier quoted context omitted.
Any chance of a technical write-up so that we can all learn from whatever happened?
Absofuckinglutely. It's being done as we speak.
Good luck explaining this
https://news.ycombinator.com/item?id=16087079
in the face of this
https://news.ycombinator.com/item?id=14905870
Literally nothing was done for 158 days. You yourself asked:
https://github.com/node-forward/discussions/issues/29#issuec...
"How would package signing prevent people from requesting the wrong package? The malware author could also sign their package."
And here is a perfect example. Someone replaced a legit package with a malicious one. Had the original author signed the package, then then NPM users could have defended against the new malicious author, because the new author's signing key would not be in their truststore.
Unsigned packages leave NPM package users defenseless. I hope that is crystal clear now.
Re: Many packages suddenly disappeared
#326Earlier quoted context omitted.
typical JavaScript engineering
Javascript is a very handy language, it's held back by all the gymnastics it needs to do to get over browser/www limitations, and an influx of low skill developers with no diploma.
Would you rather they be low skill developers with a diploma?
Re: Many packages suddenly disappeared
#327Earlier quoted context omitted.
I am very surprised that a package manager of this calibre and impact abstains from best practices when it comes to authentication through code-signing. Other package managers are miles ahead of NPM. For example, Nix, which uses immutability and hashing to always produce the same artifact, regardless of changes of the sources.
So I know rpms and debs are signed, as I've setup repos for both. Docker repositories require a valid SSL key (or you have to manually allow untrusted repos). But do Python packages and Ruby gems have signature verification? How does pypy/pip and gem deal with validating a package is what it claims to be?
PyPI (which is what Pip uses) at the very least does not require authors to sign their packages. I can't say whether it supports signing though.
Re: Many packages suddenly disappeared
#328Earlier quoted context omitted.
Yikes, what is it about node/npm/javascript that makes it feel like a house of cards? I think the (short) answer is "node, npm, and javascript". The longer answer has something to do with the automatic installation of dependencies, and the common use of shell scripts downloaded directly off the internet and executed using the developer's or sysadmin's user account. I used to use CPAN all the time. CPAN would check de…
I had very limited contact with CPAN some years ago but I imagine it was slightly more sane in terms of granularity of dependencies. Whereas a lot of npm modules are relatively small - some tiny - and have their own dependencies. So a simple "npm install blah" command can result in dozens of packages being installed. Dealing with that manually would, in fairness, be a giant chore. Now of course there's a discussion t…
Whether it's a good idea or not, that's what JS' lack of stdlib produces.
Re: Many packages suddenly disappeared
#329Yikes, what is it about node/npm/javascript that makes it feel like a house of cards?
I have recently taken over an Angular project (with a C# backend, thankfully) at my job. It took two hours to get it to even compile correctly because some dependencies were apparently outdated in package.json and it just ran on the other dev's machine by accident. I don't understand why I need over 100 dependencies for a simple Angular Single Page App that pulls JSON from the backend and pushes JSON back. Meanwhile,…
Mark, is that you?
Kidding - but we had exactly the same problem, except with a React app rather than an Angular one just before Christmas.
No joke on with this statement though: every time we have a time-consuming build issue to deal with it comes down to some npm dependency problem. Honestly, if there were a way we could realistically ditch npm (NO! YARN IS NOT ANY BETTER - to preempt that suggestion - it's simply an npm-a-like) I'd happily do so but sadly there isn't.
Re: Many packages suddenly disappeared
#330There are over 700 comments on this issue on GitHub. It is turning into live chat room. https://github.com/npm/registry/issues/255 Be part of the history :) EDIT: there are now over 1100 comments/memes.
The final count was 1263 comments, and no more will be added now that it's closed and locked.