Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

50% of new NPM packages are spam

blog.sandworm.dev

Post reply on HN
Print view View original

171–180 of 325 posts

Re: 50% of new NPM packages are spam

#171

Post by loic-sharma »

adql wrote: post #127
loic-sharma wrote: post #57

Earlier quoted context omitted.

Sure, but removing or unlisting a valid package could break projects. The folks maintaining the package ecosystem need to be careful. Let’s say there’s 10 spam uploads per hour and it takes you 1 second to verify a package is spam and remove it. That’s 30 minutes a week just dealing with spam. While I was on the .NET package manager, we had the on-call engineer handle this thankless chore. Could you detect these pack…

We're talking about packages that don't even come with code > More than half of all new packages that are currently (29 Mar 2023) being submitted to npm are SEO spam. That is - empty packages, with just a single README file that contains links to various malicious websites. Yeah once you cut the obvious they will get smarter but at least some will leave to look for other easier target. Spammers just try to find somet…

> Spammers just try to find something that ranks high in SEO and costs them nothing, if repository stops being that most will leave.

This is not true. Spammers will continue trying even if you are very good about removing spam packages. Source: worked on a package manager for 5 years.

> Most other package repositories don't have that problem to such degree

They do, you’re just not seeing it because they’re actively removing packages. That said, NPM is the largest package ecosystem and likely receives the most spam.

> Users will do the flagging for that so at least you won't have too many valid packages to verify

The trick is to have detection that’s accurate enough that you feel confident removing packages without human intervention.

Package managers have likely already built lots of tooling to detect potential spam and then bulk remove them. That’s how they manage thousands of spam removals per week in a reasonable amount of time. Nonetheless, human verification is necessary due to the “left pad problem”. This takes time due to the sheer quantity of spam.

Re: 50% of new NPM packages are spam

#172

Post by lesquivemeau »

I was expecting this article to be a promotion of their audit tool considering a thread about it was flagged as spam less than two weeks ago[1]

Turns out it indeed is. Interesting article nonetheless, but it's quite ironic that it's about spam

[1] https://news.ycombinator.com/item?id=35233877

Re: 50% of new NPM packages are spam

#173

Post by pjc50 »

miohtama wrote: post #2

Spam problems can be solved by - Cross-Internet reputation system for accounts - Small fee on submission

The fees are in the wrong direction. Submitters of good npm packages should be paid.

(The overall lack of quality control on npm is a separate question)

Re: 50% of new NPM packages are spam

#174

Post by alkonaut »

newswasboring wrote: post #8
miohtama wrote: post #2

Spam problems can be solved by - Cross-Internet reputation system for accounts - Small fee on submission

> Small fee on submission This will immediately bias the submissions only coming in from the west. Remember you can make the fee small but sometimes a person can't even pay even if they have the money. I remember having the 1000 or so rupees required for some VPS stuff when I was a teenager and not being able to pay since I didn't have a credit card. I hope we don't ever make money a barrier to open source.

> I hope we don't ever make money a barrier to open source.

Then make some other very cumbersome proof. But it's still better to cut off half the world from open source than pollute the few large software repositories with spam, which would dissuade everyone everywhere from contributing eventually. There's no problem contributing to a library from anywhere it's just that you collaborate with someone who in turn can pay the reg/anti-spam fee.

Re: 50% of new NPM packages are spam

#175

Post by MikePlacid »

criley2 wrote: post #63
FredPret wrote: post #34

Earlier quoted context omitted.

So true. It's truly sad that some people can hold tight to their cynicism even as they build up their technical skills

The people who do this are likely not American or Western European, likely not from a wealthy background, likely don't have access to high end tech jobs, and probably can't even make 5% of what a Facebook or Google employee makes. These people might feel spite and anger towards the western world for the extreme lavish excess that developers enjoy. It's not hard to imagine a world where developers can learn some skill…

> These people might feel spite and anger towards the western world for the extreme lavish excess that developers enjoy.

Oh, let me tell you my “lived experience” of spite and anger that I once felt towards western developers.

So, it was late 1990s and our sales guys got hold of a presentation paper that competitor guys gave to a customer that both our companies were trying to win. I never read such a collection of blatant lies in my life! And I came from a one-Party country where newspapers were… uhm notorious for their lying. But not like this! Specifically a feature that I’ve spent more than half a year on, and which we were proudly shipping - was marked as not existent. Imagine somebody trying to scratch half a year of your life, and a rather intense half a year to that - out of existence. With black, lying ink.

And I clearly remember sitting and thinking: why are they doing this? The competitor was a well-established company, long time in business, probably employed citizens, provided them with pension funds and other perks - why don’t they compete with us, mostly new emigrants on a work visas - why can’t they compete on _merits_? They have everything to just sit, work and compete - why lie?

Yes, I was feeling spite and anger, true.

But, about 20 years later, just around that your famous President inauguration - this exact competitor went bankrupt. The stopping point for a buyer was - they did not want to fund pensions 100%. It was like watching Karma working right and clear in this material world - a rare moment, no?

Re: 50% of new NPM packages are spam

#177

Post by Georgelemental »

smallerfish wrote: post #147
wruza wrote: post #22

Just think of it, there is a real developer who decided to do this. Spam is immoral, but doing that to an open source repository is your personal all time low.

I think "immoral" is a reach as a description of spam, and to be crystal clear I'm not defending spam. How is spam any more immoral than ads in a web page? Both are inserting advertising into a channel that a user is accessing information through, as a way to raise revenue or change behavior. (Spam is not by definition phishing, any more than banner ads are innately phishing, though phishing can be served through bot…

You are free to put ads on your own service, because you own it and can do what you want with it. But you don't have the right to vandalize someone else's service with spam.

Re: 50% of new NPM packages are spam

#178

Post by codedokode »

criley2 wrote: post #63
FredPret wrote: post #34

Earlier quoted context omitted.

So true. It's truly sad that some people can hold tight to their cynicism even as they build up their technical skills

The people who do this are likely not American or Western European, likely not from a wealthy background, likely don't have access to high end tech jobs, and probably can't even make 5% of what a Facebook or Google employee makes. These people might feel spite and anger towards the western world for the extreme lavish excess that developers enjoy. It's not hard to imagine a world where developers can learn some skill…

While in Russia talented developers make less than a newbie developer in the West earns, their salaries are relatively high compared to non-IT jobs. You won't die in the street if you are a developer. The reason why those people spam is either because they have low technical skills and cannot find a decent job (most probably) or simply because they believe that work is for losers; successful men take money from others instead of working like a slave.

As they lure people into Telegram channels in hope to scam them, I assume that the conversion is low and this is not very profitable and they do this because of lack of skills.

Re: 50% of new NPM packages are spam

#179

Post by lookdangerous »

Nextgrid wrote: post #87
lookdangerous wrote: post #83

Earlier quoted context omitted.

How about instead of who owns it, ask who uses it?

I don't think this would affect most developers? The value of NPM is a host of packages that you reference in package.json, not its web UI. The spam on the web UI is dangerous for victims that land there via search engines, but I don't think this would affect NPM's actual users that much?

Thanks for clarifying the situation

Re: 50% of new NPM packages are spam

#180

Post by onion2k »

sbarre wrote: post #168
onion2k wrote: post #144

Earlier quoted context omitted.

Coz you might want results not only from docs but stackoverflow and other places ? Of course there is. I want docs, examples, and maybe opinions vs alternatives if I look to solve problem X with external dependency. You don't need the link to the package in NPM to be in the results for either of these examples.

You're trying real hard to tell people how they _shouldn't_ be doing their work, maybe accept that your opinion, while valid, is just that - your opinion - and that others have their own equally valid ways of approaching their work and their searching?

I'm suggesting that it wouldn't be a problem if NPM switched off indexing, and if the article is correct that half of packages are spam then it'd actually be significantly beneficial.

The broader point is that by expecting Google to be a single interface to the entire internet, and refusing to accept that there might be some places you need to go to directly, we make the problem of spam worse. Using Google for navigation when you know what site you want rather than using that site's search feature incentivizes spammers to abuse things they would otherwise ignore.

Post reply on HN