Live data from Hacker News

Many packages suddenly disappeared

github.com

51–60 of 520 posts

Re: Many packages suddenly disappeared

#51
post #34

I don't understand much about the blockchain, but one thing I have heard is that it's impossible (or very hard) to remove things from it. It is immutable, sort of append only, if I understand it correctly. So my question is, is there anyone working on moving npm to the blockchain? Or doing something like a package manager on the blockchain? If not, why not?

I don't want to have to download 135 gigabytes to require left-pad.

Good thing you wouldn't ever need to do such a thing.

Re: Many packages suddenly disappeared

#52
post #44

Btw. for those who don't know: Yarn (which is an alternative to npm) uses a global cache [1] on your machine which speeds things up, but probably also protects you from immediate problems in cases like the one currently on progress (because you would probably have a local copy of e.g. require-from-string available). [1] https://yarnpkg.com/lang/en/docs/cli/cache/

Already counting down the days before yarn is considered old and broken and people are recommending switching to the next hot package manager/bundler...

Re: Many packages suddenly disappeared

#53
post #17

Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Relying on npm, Atlassian/GitHub etc really hurts when stuff like this happens. Issues always gets resolved, but cases such as the GitLab incident should be enough to always keep some local copies around.

> Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Install an instance of Sonatype Nexus, create a proxy-repo for npm (and Maven if you also use Java) and that's it. What, however, won't be caught is Docker (because that crap insists on directly talking to the Dockerhub servers, which is a giant security hole waiting to happen) and PHP com…

Thanks for this, will look into it.

Re: Many packages suddenly disappeared

#54
post #17

Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Relying on npm, Atlassian/GitHub etc really hurts when stuff like this happens. Issues always gets resolved, but cases such as the GitLab incident should be enough to always keep some local copies around.

> Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Install an instance of Sonatype Nexus, create a proxy-repo for npm (and Maven if you also use Java) and that's it. What, however, won't be caught is Docker (because that crap insists on directly talking to the Dockerhub servers, which is a giant security hole waiting to happen) and PHP com…

You can setup mirrors for dockerhub... Or any docker registry.

You also can require image signing such that if an image is signed by an untrusted party it will fail.

Re: Many packages suddenly disappeared

#55
post #34

I don't understand much about the blockchain, but one thing I have heard is that it's impossible (or very hard) to remove things from it. It is immutable, sort of append only, if I understand it correctly. So my question is, is there anyone working on moving npm to the blockchain? Or doing something like a package manager on the blockchain? If not, why not?

You don't need a full blockchain for this: the relevant property is (somewhat tautologically) that it's an append-only data structure. By convention, everyone processing the blockchain looks to make sure that the new blockchain they get is a descendant of the previous blockchain they already have.

There are lots of other structures that work like this. Git is one - when git fetches a branch, git will check whether the remote branch includes all the commits it saw last time it fetched, or some are missing. (By default this is non-fatal but tends to produce warnings/errors when you try to actually use the replaced branch, but you can easily make this fatal.) Another good one is the style of Merkle trees used in Certificate Transparency: there's no proof-of-work, so the trees are small, but they still include a cryptographic hash of each previous tree so you can detect if something has gone missing.

The other relevant property of the blockchain is that it's not a reference to data elsewhere, it (like git) actually contains all the data that's ever been on the blockchain, and you need that data to verify the blockchain properly. This may or may not be what you want for a programming language package manager; it means that in order to set up a new development environment, you have to download every version npm package that ever existed. It does accomplish the goal of preventing things from being removed, but it's pretty heavy-weight.

Re: Many packages suddenly disappeared

#56
post #34

I don't understand much about the blockchain, but one thing I have heard is that it's impossible (or very hard) to remove things from it. It is immutable, sort of append only, if I understand it correctly. So my question is, is there anyone working on moving npm to the blockchain? Or doing something like a package manager on the blockchain? If not, why not?

That is a smart idea. but it would cost a lot to publish a package.

Re: Many packages suddenly disappeared

#58
post #12
post #5

PSA: Please be cautious because this is an excellent opportunity for taking over packages and injecting malware by malicious people. Example: https://www.npmjs.com/package/duplexer3 which has 4M monthly downloads just reappeared, published by a fresh npm user. They published another two versions since then, so it's possible they've initially republished unchanged package, but now are messing with the code. Previously…

didn't npm make some changes where a published package name cannot be republished, at least not without npm intervention?

I thought so too. I thought they did that after the left pad incident.

Re: Many packages suddenly disappeared

#59
post #17

Gah. Moments like these always gives me a bit of panic, since I realize that so much of my software relies on external sources. Relying on npm, Atlassian/GitHub etc really hurts when stuff like this happens. Issues always gets resolved, but cases such as the GitLab incident should be enough to always keep some local copies around.

I've stopped wondering about NPMs structure. But still: Our bog-standard in-house java development setup would be unaffected by this class of problems. You need some kind of private maven repository, and nexus or artifactory automatically mirrors downloaded dependencies. And on top of that, versions are pinned per default. So new malicious versions wouldn't be used either. We could safely build new hotfix releases even with maven central 100% down or compromised.

Granted, we do depend on bitbucket. However, I am honestly scared to self-host our code. This is a small but old shop, so the entire code base is easily several million dollars worth in man-hours alone. And then again, it's git, so if push comes to shove, we could easily and quickly spin up an internal gitlab instance and push our stuff there to get back up.

Re: Many packages suddenly disappeared

#60
post #48

Earlier quoted context omitted.

Does "stalinist" work?

Dunno, but it's quite hard to resist childish urges. I note that "jew" is still available, and .jew is a perfectly valid extension which no program has ever claimed. I love all cultures. There's something worth learning from each of them, if you try. But emergent social dynamics are deeply interesting too: I get it (everyone does), and it makes perfect sense. Yet we've all collectively agreed that those are unwholeso…

> But HN user "baby" recently made a blockchain based image board that was theoretically impossible to control. How would people behave in such a situation? Who would you even punsh?

Probably will get scooped up by pedophiles sooner or later, followed by a couple high profile arrests and everyone will be scared shitless to run a node for that blockchain. CP has always been a good source of scare for Tor exit node operators, thankfully enough Tor nodes don't store the stuff in contrast to a blockchain - so at least the operators didn't have to serve jail time but a couple have had their houses raided by the cops and all IT equipment confiscated for months.

Actually, from a libertarian POV I really like the idea, both behind an uncensorable imageboard and Tor, but the simple fact that pedophiles and Nazis can and will abuse the openness for their vile gains makes me believe that the world will probably never be ready for such a thing as widespread service. For now we as society have to be lucky that many pedophiles, drug dealers and Nazis don't really care about good opsec... but that one is bound to change.

Post reply on HN