Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

50% of new NPM packages are spam

blog.sandworm.dev

Post reply on HN
Print view View original

121–130 of 325 posts

Re: 50% of new NPM packages are spam

#121

Post by dpkirchner »

wongarsu wrote: post #28
hollowturtle wrote: post #11

I'm afraid it can get worse. What happens when there will be a proliferation of "looking legit npm packages" thanks to AI, full with ransomware? Currently I can't really figure out a one size fits all solution to that. Any idea?

One idea that's gaining (marginal) traction in Rust (which really sits in the same boat here) is trusted reviews, where trust is established by a web of trust. You probably have some developers you trust, and they have a different set of people they trust, so you can establish transient trust (that decays as the chain gets longer). The most relevant project for Rust is https://web.crev.dev/rust-reviews/ , not sure if…

This sounds good. Seems like the easiest way to start is to use the package.json-defined dependencies to create the web/tree. If a developer of package A use package B, they trust the developer of package B, and so on.

Re: 50% of new NPM packages are spam

#122

Post by alexb_ »

criley2 wrote: post #116
ozim wrote: post #100

Earlier quoted context omitted.

No he means “being poor isn’t an excuse for being asshole”. Just like keying your neighbor car because he could afford nice one is not acceptable whatever you feel like.

"Keying your neighbors car because they have a nicer one" is not an analogy that works for anything here. What is happening in NPM is not a car being keyed. There is a profit motive for doing this. Perhaps you could say "Stealing 1 gallon of gas from your rich neighbors car to feed your starving children makes you an asshole", that's an analogy that seems to fit what is happening here, and an opinion I would disagree…

What is your opinion on catalytic converter thieves?

Re: 50% of new NPM packages are spam

#123

Post by eddieroger »

wruza wrote: post #22

Just think of it, there is a real developer who decided to do this. Spam is immoral, but doing that to an open source repository is your personal all time low.

You don't know what circumstances the other party, the spammer, is under in this situation. On one end, maybe they just don't care, which is certainly their choice. Maybe this is the difference between eating tonight or not, or feeding their family. We may think it's immoral, but those are in the light of our own circumstances.

Re: 50% of new NPM packages are spam

#124

Post by justinclift »

pipo234 wrote: post #106
mindcandy wrote: post #72

Earlier quoted context omitted.

In the real world, do wealthy companies want to be named on this list of 3 or 4 groups spamming NPM? That’s a lot different than being seen buying a banner ad.

That's probably where the reputation part would come in - fair enough. Still, a large wealthy company might consider creating an "unaffiliated" front company to act on their behalf. For example, take out a legit open source competitor by having the front publish a mediocre bogus project with very similar name. Or paying a small fee to bundle malware into a legit FOSS. So, similar to Twitter's blue check mark - Yes, a…

Most of that seems _very_ theoretical, with the exception of "paying to bundle malware into legit FOSS".

Unfortunately, that last one deserves a special Fuck You to the main developers of FileZilla, who have knowingly bundled malware for years. :(

For anyone that doesn't know about it, here's a forum thread about it they haven't deleted:

https://forum.filezilla-project.org/viewtopic.php?t=50565

Re: 50% of new NPM packages are spam

#125

Post by 11235813213455 »

Kye wrote: post #120
11235813213455 wrote: post #119

Earlier quoted context omitted.

as a developer you can also keep a relatively low number of dependencies, and mainstream or simple ones

That takes awareness and discipline. The last time I tried to learn Node, all the guides led you down a road of dependency hell.

that takes experience, like everything you want to do well

Re: 50% of new NPM packages are spam

#126

Post by xenophonf »

criley2 wrote: post #63
FredPret wrote: post #34

Earlier quoted context omitted.

So true. It's truly sad that some people can hold tight to their cynicism even as they build up their technical skills

The people who do this are likely not American or Western European, likely not from a wealthy background, likely don't have access to high end tech jobs, and probably can't even make 5% of what a Facebook or Google employee makes. These people might feel spite and anger towards the western world for the extreme lavish excess that developers enjoy. It's not hard to imagine a world where developers can learn some skill…

My (former) friends who built thousands of websites to manipulate pagerank back in the day were definitely wealthy westerners purposefully gaming the system to make even more money for themselves, to the detriment of the rest of us.

Re: 50% of new NPM packages are spam

#127

Post by adql »

loic-sharma wrote: post #57
supriyo-biswas wrote: post #23

Is this spam not easily mitigated by simple Bayesian approaches and collection of link features by visiting them?

Sure, but removing or unlisting a valid package could break projects. The folks maintaining the package ecosystem need to be careful. Let’s say there’s 10 spam uploads per hour and it takes you 1 second to verify a package is spam and remove it. That’s 30 minutes a week just dealing with spam. While I was on the .NET package manager, we had the on-call engineer handle this thankless chore. Could you detect these pack…

We're talking about packages that don't even come with code

> More than half of all new packages that are currently (29 Mar 2023) being submitted to npm are SEO spam. That is - empty packages, with just a single README file that contains links to various malicious websites.

Yeah once you cut the obvious they will get smarter but at least some will leave to look for other easier target.

Spammers just try to find something that ranks high in SEO and costs them nothing, if repository stops being that most will leave. Most other package repositories don't have that problem to such degree

> unlisting a valid package could break project

... and about packages that most likely are NOT used as dep anywhere

> Let’s say there’s 10 spam uploads per hour and it takes you 1 second to verify a package is spam and remove it. That’s 30 minutes a week just dealing with spam. While I was on the .NET package manager, we had the on-call engineer handle this thankless chore.

No need. Just add flag button where a package can be flagged for a check. Users will do the flagging for that so at least you won't have too many valid packages to verify

> Could you detect these packages at upload time? Yes, but spammers will change their patterns once the package ecosystem gets too effective at detecting current patterns. Perhaps machine learning could help, but often times package manager teams are small and don’t have expertise in this area.

With AI I'm afraid it might get awfully close to "newbie user just publishing package full of shit code"

Re: 50% of new NPM packages are spam

#128

Post by Dudeman112 »

themitigating wrote: post #79
criley2 wrote: post #63

Earlier quoted context omitted.

The people who do this are likely not American or Western European, likely not from a wealthy background, likely don't have access to high end tech jobs, and probably can't even make 5% of what a Facebook or Google employee makes. These people might feel spite and anger towards the western world for the extreme lavish excess that developers enjoy. It's not hard to imagine a world where developers can learn some skill…

Being jealous isn't a justification for any action

Ah, the age-old mixing pointing out the reasons for why an individual might act they way they do with morally absolving them

Ever common amongst people who have never seen or felt the consequences of abject poverty

Re: 50% of new NPM packages are spam

#129

Post by millerm »

vidyesh wrote: post #95
wruza wrote: post #22

Just think of it, there is a real developer who decided to do this. Spam is immoral, but doing that to an open source repository is your personal all time low.

I don't think this kind of spam is new. Its just your perspective that determines this is immoral . An argument can be made that any tool built to gain SEO advantage is also borderline immoral and those tool exists for almost a decade now. There are and have been bots to generate SEO content and/or spam websites and custom plugins for Wordpress which achieve that. All to game the search engine. This too is immoral as…

Many years ago I quit my job at a search engine company for my personal ethics, because they had me start manipulating search results based on who paid for their entries.

Re: 50% of new NPM packages are spam

#130

Post by pmontra »

verdverm wrote: post #73

Is this outcome a point against having centralized registries? Why not go straight to the source code host?

This is basically what I was doing in the 80s and 90s, downloading compressed tarballs from ftp sites and compiling them. It takes quite a longer developer time than the package manager approach. That includes the time to learn which sites you can trust (probably none today) and which dependencies to use (usually listed in the README.) Furthermore there would be a big incentive to use very few libraries: this is both good and bad. Good because there won't be silly one function modules, bad because a dozen of small modules can add significant value to a project in a short time. Having to code them or build a bigger all comprising module is much harder.
Post reply on HN