Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

70% of new NPM packages in last 6 months were spam

blog.phylum.io

Post reply on HN
Print view View original

41–50 of 116 posts

Re: 70% of new NPM packages in last 6 months were spam

#41

Post by 3np »

arp242 wrote: post #39
3np wrote: post #29

Earlier quoted context omitted.

No, it's more than that. Did you read the documentation page linked in the comment you replied to? > But what about after the command has run? If you munge around in node_modules after a successful `npm ci`, that's on you. If you run scripts that do, that's on you. If you depend on packages that run such scripts, that's on you. > What, you mean I'm supposed to audit my dependencies myself? That's too much work! Yes,…

It says exactly that: "If a node_modules is already present, it will be automatically removed before npm ci begins its install". I didn't "munge around" in node_modules; I said "if something goes wrong". Like I said in my previous comment: "I found this out a few ago after I had some corrupted files due to a flaky internet connection" . That's not munging around, that's computers being computers. Network errors happe…

Well, you did say (emphasis mine):

> That's just "rm -r node_modules && npm install".

And there is more to it, very much relevant to your original complaint, which is not applicable to it.

> I didn't "munge around" in node_modules; I said "if something goes wrong". Like I said in my previous comment: "I found this out a few ago after I had some corrupted files due to a flaky internet connection".

I do believe that if you had run `npm ci` instead of `npm install`, that would have resulted in an error and an empty node_modules instead of inconsistency.

> Disk errors happen. Memory errors happen. Things like that.

Those are at lower layers than the package manager. I think it's unreasonable to expect the package manager to check for inconsistencies induced by hardware errors. You have file-system level solutions (zfs,btrfs) and things like ECC for that. Nothing is a silver bullet.

> I've also had an install ISO corrupted at some point. I always check the sums since, just in case because there was a lot of confusion involved before I found out the ISO was just downloaded wrong for some reason

Good! It happens. Keep doing it. There's a reason integrity verification is a step in every Linux distro installation guide.

> Your entire post is a masterclass in arguing against things that were never claimed and forceful injection of your own bugbears.

Fair! Though I do want to call out your "Who is carefully auditing if the repo URL in the lockfile is actually the correct one?" in a sibling comment - apparently the inferred bugbear wasn't all off-base :p

> I just want to check if node_modules is identical to what's expected, just in case, because computers kind of suck and are unreliable.

I think we can agree on that npm is not sufficient tooling for this. My response is additional tooling. I guess you want a more feature-complete package manager. Wish I had one to recommend. npm is severely neglected after the MS acquisition; yarn maintainers are completely misguided on a couple of fundamentals; every time I get around to take another look at pnpm either I run into a bug or catch a recent enough breakage-outside-of-semver that I decide they're not ready yet...

Re: 70% of new NPM packages in last 6 months were spam

#42

Post by tyingq »

arp242 wrote: post #40
tyingq wrote: post #32

Earlier quoted context omitted.

I think that's true for most package managers. That if there's a lock file, there's typically a default command to use it for installs and ignore the main config file.

Yeah maybe, I don't really know off-hand and I'd have to check. I know it's not possible in Go but not sure about anything else. I'd consider it hugely surprising for other packagers where that's possible too. Who is carefully auditing if the repo URL in the lockfile is actually the correct one?

Poetry for sure acts this way. Some checks on things like "poetry.lock is older than pyproject.toml", but no real checks unless you specifically ask for them. Not saying it's good, of course. Just that it's typical.

Re: 70% of new NPM packages in last 6 months were spam

#43

Post by nixnixers »

n_ary wrote: post #13

Why are these spam accounts not perma banned and removed? For example, this[1] account mentioned in the article has 1781 packages of gibberish. Also, the whole reporting process is onerous, there is a large form. Of course, gatekeeping on reporting is good, but there should be a possibility to report an entire profile of package publisher. [1] https://www.npmjs.com/~eleanorecrockets

Isn't it better to leave accounts that correlate spam than to force spammers to obscure the connection by creating a new account for each piece of spam?

Re: 70% of new NPM packages in last 6 months were spam

#44

Post by dartos »

falcor84 wrote: post #23
marcus_holmes wrote: post #16

> Next, because the AI hype train is at full steam, we must point out the obvious. AI models that are trained on these packages will almost certainly skew the outputs in unintended directions. These packages are ultimately garbage, and the mantra of “garbage in, garbage out” holds true. hmm, inspiring thoughts. An answer to "AI is going to replace software developers in the next 10 years" is to create 234876238562856…

That's not what "hallucination" is. Hallucinations in LLMs are when they unexpectedly and confidently extrapolate outside of their training set when you expected them to generate something interpolated from their training set. In your example that's just a pollution of the training set by spam, but that's not that much of an issue in practice, as AI has been better than humans at classifying spam for over a decade no…

There’s just so much wrong here.

So many mangling of meaning.

Like the “AI” that detects spam is way different than LLMs.

Re: 70% of new NPM packages in last 6 months were spam

#46

Post by dartos »

arp242 wrote: post #12
vorticalbox wrote: post #6

Earlier quoted context omitted.

A "better" way is to modify the package-lock.json. You can still spoof the package but almost no one actually reviews it as npm will usually modify 1000s of lines. for example take mongoose "resolved": "https://registry.npmjs.org/mongoose/-/mongoose-8.4.4.tgz", "integrity": "sha512-Nya808odIJoHP4JuJKbWA2eIaerXieu59kE8pQlvJpUBoSKWUyhLji0g1WMVaYXWmzPYXP2Jd6XdR4KJE8RELw==", so long as the integrity check passes for the…

Hugely surprising that package.json and package-lock.json don't have to match. The way I would expect it to work is something like: for d in dependencies_from_package_json() get_package(d) if hash_package(d) != package_lock_hash(d) error() end end And not: use_package_lock_and_ignore_package_json_lol_fuck_you_haha_kthxbye() I also discovered that npm doesn't actually verify what's in node_modules when using "npm inst…

FWIW l, I believe “npm ci” is the install command which strictly respects the lock file

Re: 70% of new NPM packages in last 6 months were spam

#47

Post by _heimdall »

bigDinosaur wrote: post #30
3np wrote: post #24

Earlier quoted context omitted.

That would be a clear violation of the npm Unpublish Policy[0]. If all it takes is some spam and pissing people off to walk away from principles, they never meant anything. A proper response needs to not break expectations like this. [0]: https://docs.npmjs.com/policies/unpublish

The entire NPM ecosystem is a garbage fire. Who cares about whatever 'principles' it supposedly has? Other than avoiding malware I can't think of something I care about less than whatever principles NPM / JS developers in general have because they've mostly been bad so far. I wouldn't be surprised if principles in this case leave us with thousands of spam packages degrading the node ecosystem forever. It'd be exactly…

Its not about principles in some abstract sense though, its terms if use. Package authors need to know what the rules of the road are when dedicating time to publishing to npm, and package users need to know how much they can rely on the packages they depend on still being there tomorrow.

It'd be one thing if npm added audit warnings along the lines of "3 dependencies are likely spam." It'd be a totally different story for npm to remove them automatically based on a toolset used, in the GP example.

Re: 70% of new NPM packages in last 6 months were spam

#48

Post by yas_hmaheshwari »

renegat0x0 wrote: post #17

I am really interested if that really matters. Package managers often comes with rating system. npmjs has weekly downloads, pull requests, and other popularity scores. I am layman in AI, but why would anyone think that this would affect anything, like AI? Why would anyone train on noname package, that noone uses? Stats for spam packages can have higher-than-none stats, but that also makes them vulnerable for sweep re…

Agree! Not only in companies, but I have never seen anyone download a package, without looking at Github stars

The real fun would happen if the next incentive is to publish a package and get Github stars for that repo :-)

Re: 70% of new NPM packages in last 6 months were spam

#49

Post by arp242 »

vb-8448 wrote: post #5

I wonder what is the long term plan. Maybe the next step is to sell the control of all these packages to a rogue entity to be used for a supply chain attack?

Who says there is one? It takes basically zero effort to publish these packages, so why not do it? Script kiddie stuff. Lots of people run dumb unsuccessful hustles. The long term plan seems to be macaroni. That is: throw enough macaroni at a wall and hopefully some of it will stick. Or maybe not. Who cares? Wasn't my macaroni and I won't have to clean the wall.

Re: 70% of new NPM packages in last 6 months were spam

#50

Post by meiraleal »

nixnixers wrote: post #43
n_ary wrote: post #13

Why are these spam accounts not perma banned and removed? For example, this[1] account mentioned in the article has 1781 packages of gibberish. Also, the whole reporting process is onerous, there is a large form. Of course, gatekeeping on reporting is good, but there should be a possibility to report an entire profile of package publisher. [1] https://www.npmjs.com/~eleanorecrockets

Isn't it better to leave accounts that correlate spam than to force spammers to obscure the connection by creating a new account for each piece of spam?

That's not how spammers work. There is this profile with thousands and there are still hundreds of spam profiles with just a handful of packages yet. If you let them grow unchecked, they grow, exponentially. The broken Windows theory fits well here
Post reply on HN