Would be interesting to see the load curves on github and npm servers due to people and servers retrying downloads.
Many packages suddenly disappeared
91–100 of 520 posts
Re: Many packages suddenly disappeared
#92Hmm, I Java world we pretty much always used a local (company-owned) Maven proxy server, which grabbed packages from public repos and cached them locally to make sure builds still work if public servers were down or slow... or packages disappeared. This isn't a standard practice in JS world?
We use Nexus at work and have linked up NPM to it along with several Maven repos. I don't know why anyone wouldn't do this if they were a business.
Re: Many packages suddenly disappeared
#93I don't understand much about the blockchain, but one thing I have heard is that it's impossible (or very hard) to remove things from it. It is immutable, sort of append only, if I understand it correctly. So my question is, is there anyone working on moving npm to the blockchain? Or doing something like a package manager on the blockchain? If not, why not?
You don't need a full blockchain for this: the relevant property is (somewhat tautologically) that it's an append-only data structure. By convention, everyone processing the blockchain looks to make sure that the new blockchain they get is a descendant of the previous blockchain they already have. There are lots of other structures that work like this. Git is one - when git fetches a branch, git will check whether th…
> By convention, everyone processing the blockchain looks to make sure that the new blockchain they get is a descendant of the previous blockchain they already have.
This is not true. Bitcoin-esque blockchains are NOT append only. The only thing clients do is: 1) ensure the blockchain they have received is valid, and 2) that it is longer (more total PoW). If those conditions are met, they will consider that new chain the current chain.
You can come up with a new, completely different sequence of blocks, send it to clients on the network, and get them to start using that new chain instead as long as it has more PoW.
Re: Many packages suddenly disappeared
#94Earlier quoted context omitted.
Or just don't .gitignore node_modules, then diff any changes to node_modules on update.
Does not work as soon as you use node modules that come with native components that have to be recompiled for the machine, and there are many of these. Colleagues have been bitten by this - one used OS X 10.11, the other 10.12, and they experienced weird bugs from this. Went away once they kicked out node_modules from git.
Yeah, it’s an annoying problem. Maybe you could gitignore the *.node (the native module file extension) files only. But I’m not sure how you’d rebuild those “on demand” after a checkout without running 'npm install' from the top level.
Re: Many packages suddenly disappeared
#95NPM is extremely vulnerable to typosquatting. Be cautious with what you install. The install scripts can execute arbitrary code. NPM's team response is that they hope that malicious actor won't exploit this behaviour. According to my tests, typosquatting 3 popular packages allows to take over around 200 computers in 2 weeks time it takes their moderators to notice it.
this is not a response? http://blog.npmjs.org/post/168978377570/new-package-moniker-...
That's okay, but it's not enough - it's easy to swap two letters and do similar substitutions to fool many users. If a package is downloaded 10,000 times every day, surely once in a while someone will misspell the name somehow.
Other than that, their reaction to similar incidents was to wait for somebdoy on twitter to notify them, ban the responsible users, and hope that it won't happen again. It's still extremely exploitable and there are surely many other novel ways of installing malware using the repository that we haven't even heard of yet. The NPM security team is slow to act and sadly doesn't think ahead. They're responsible for one of the largest software ecosystems in the world, they should step up their game.
Re: Many packages suddenly disappeared
#96So, funny story: I registered the "nazi" npm package. When you require it, it says "I did nazi that coming." That's it. (Though it would've been a funny name for a linter.) ... Or it did. I received a harshly worded letter from npm saying they axed it. It hit all the talking points about inclusiveness and making sure no one feels even slightly annoyed. Meh. No point to this story. Just an interesting situation with a…
They document that they curate package names here: https://www.npmjs.com/policies/conduct I guess lots of people will think that a policy like Avoid using offensive or harassing package names, nicknames, or other identifiers that might detract from a friendly, safe, and welcoming environment for all. stifles their inner something or other though.
seems like a good reflection of the current social climate: they have a policy to prevent mildly offensive package names, and they enforce it, but they don't have a solution to packages randomly disappearing and being replaced with malicious versions.
Re: Many packages suddenly disappeared
#97Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Relying on npm, Atlassian/GitHub etc really hurts when stuff like this happens. Issues always gets resolved, but cases such as the GitLab incident should be enough to always keep some local copies around.
Having additional copies is always a good idea, but you already get that by just installing the modules on developers' machines.
At some point you have to trust a third party. Even if you run your own hardware, you still depend on power and internet provided by someone else. And unless you are a massive company, time is typically spent much better on other things than hosting your own NPM packages and git repos.
Re: Many packages suddenly disappeared
#98Re: Many packages suddenly disappeared
#99In my org, we use Artifactory as a cache between us and external sources. They have a free version too. I'd encourage everyone to use it, or something like it. Stop pointing your package managers to the public registry.
Minutes before reading this comment I sent an email to our team to verify our artifactory did not download any packages over the weekend. Most likely not as nobody is working (to add a new one or update a version) but better safe than sorry. +1 for artifactory cache.
Re: Many packages suddenly disappeared
#100Glad we cache ours on ProGet now!
Not a ProGet user here, but this definitely seems like a good idea. What are the open-source / self-hosting options here? It gets a little messy with all the sub-dependencies, doesn't it?
If you only need NPM support we are very happy with npm-register[1]. I also heard good things about verdaccio[2].
Plus you also get private packages.