70% of new NPM packages in last 6 months were spam • Hacker News Board

Re: 70% of new NPM packages in last 6 months were spam

#101

Post by louislang » Wed, Aug 07, 2024, 4:30 PM UTC

NikxDa wrote: post #2

> Contrary to what npm states, this package actually depends on one of our aforementioned spam packages. This is a by-product of how npm handles and displays dependencies to users on its website. For me personally, this is the biggest surprise and takeaway here. By simply having a key inside package.json's dependencies reference an existing NPM package, the NPM website links it up and counts it as a dependency, regar…

(Full disclosure: I'm one of the co-founders @ Phylum)

We could do a full write-up on npm's quirks and how one could take advantage of them to hide intent.

Consider the following from the post's package.json:

    "axios": "https://registry.npmjs.org/@putrifransiska/kwonthol36/-/kwonthol36-1.1.4.tgz"

Here it's clear that the package links to something in a weird, non-standard way. A manual review would tell you that this is not axios.

The package.json lets you link to things that aren't even on npm [1]. You could update this to something like:

    "axios": "git://cdnnpmjs.com/axios"

And it becomes less clear that this is not the thing you were intending. But at least in this case, it's clear that you're hitting a git repository somewhere. What about if we update it to the following?

    "axios": "axiosjs/latest"

This would pull the package from GitHub, from the org named "axiosjs" and the project named "latest". This is much less clear and is part of the package.json spec [2]. Couple this with the fact that the npm website tells you the project depends on Axios, and I doubt many people would ever notice.

[1] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#...

[2] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#...

Re: 70% of new NPM packages in last 6 months were spam

#102

Post by johnmw » Wed, Aug 07, 2024, 4:33 PM UTC

I was sad to read this and thought "this is why we can't have nice things."

But following the links was fun and educational:

"The end goal here [of the Tea protocol] is the creation of a robust economy around open source software that accurately and proportionately rewards developers based on the value of their work through complex web3 mechanisms, programmable incentives, and decentralized governance."

Which lead to:

"The term cobra effect was coined by economist Horst Siebert based on an anecdotal occurrence in India during British rule. The British government, concerned about the number of venomous cobras in Delhi, offered a bounty for every dead cobra. Initially, this was a successful strategy; large numbers of snakes were killed for the reward. Eventually, however, people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped. When cobra breeders set their snakes free, the wild cobra population further increased."

Which lead to:

"Goodhart's law is an adage often stated as, 'When a measure becomes a target, it ceases to be a good measure.'"

Re: 70% of new NPM packages in last 6 months were spam

#103

Post by nixnixers » Wed, Aug 07, 2024, 5:11 PM UTC

esprehn wrote: post #71

nixnixers wrote: post #43

Earlier quoted context omitted.

Isn't it better to leave accounts that correlate spam than to force spammers to obscure the connection by creating a new account for each piece of spam?

That primarily works if you can shadow ban the account. Otherwise the spam is still negatively impacting the community (ex. By polluting search results).

If you make them create a new account each time you remove a package, how does that help you find or remove pollution going forward? It seems to work in the moment, but if you have no plan to change the system the resulting equilibrium is worse than if you can identify a connection between packages from the same spammer.

Re: 70% of new NPM packages in last 6 months were spam

#104

Post by fennecbutt » Wed, Aug 07, 2024, 5:15 PM UTC

I mean realistically it's representative of the Internet as a whole. Makes me wonder where all the porn packages are.

The pulling in of unexpected dependent packages is a real issue though, how do other ecosystems deal with it? NPM is really missing some level of trust beyond just using "brand name" packages.

My general judgement is usually how often it's worked on/how many downloads it has but gut feel isn't really enough, is it?

Re: 70% of new NPM packages in last 6 months were spam

#105

Post by Too » Wed, Aug 07, 2024, 5:25 PM UTC

renegat0x0 wrote: post #17

I am really interested if that really matters. Package managers often comes with rating system. npmjs has weekly downloads, pull requests, and other popularity scores. I am layman in AI, but why would anyone think that this would affect anything, like AI? Why would anyone train on noname package, that noone uses? Stats for spam packages can have higher-than-none stats, but that also makes them vulnerable for sweep re…

If you look at the purpose of this Tea protocol it is exactly to provide a chain of credibility. Though, by connecting ranking with monetization, tea has created perverse incentives, leading spammers to pump up their tea ranking, by linking and starring packages in circles. Their goal is to make it look like it’s a highly used package.

Luckily, nobody thinks that tea ranking matters, except for the spammers themselves.

They are with no doubt attempting to poke at other more established metrics as well. This could eventually fool an AI or even humans.

Re: 70% of new NPM packages in last 6 months were spam

#106

Post by brynb » Wed, Aug 07, 2024, 5:49 PM UTC

lupire wrote: post #87

brynb wrote: post #79

Earlier quoted context omitted.

it's language-cultural. to "publish a package" in Go simply means having a public git repository. and yet, nobody who writes Go imports packages. it's well-understood that if you can't write something like leftpad (or many other JS packages) yourself in your own codebase in a few lines, you're an absolute nonce. Javascript developers on the other hand tend to skew towards the juniors in our broader ecosystem, and the…

So no one uses all those Go libraries on GitHub? Hmmm except pedophiles ? What is wrong with you?

... _what_? i don't think you parsed my comment correctly at all

to address the part of your comment that doesn't make my head spin: only very occasionally do i see senior Go developers import 3rd party libraries. i'm just speaking from my experience

Re: 70% of new NPM packages in last 6 months were spam

#107

Post by Fatnino » Wed, Aug 07, 2024, 6:12 PM UTC

There was a similar thing to tea a while back. I think I saw the project posted on here. Went to their github and found a typo in their Readme. Opened a pr with a correction and then they started sending me about a dollar in btc every month till they ran out of money and the project imploded.

Re: 70% of new NPM packages in last 6 months were spam

#108

Post by falcor84 » Wed, Aug 07, 2024, 10:00 PM UTC

ffsm8 wrote: post #27

falcor84 wrote: post #23

Earlier quoted context omitted.

That's not what "hallucination" is. Hallucinations in LLMs are when they unexpectedly and confidently extrapolate outside of their training set when you expected them to generate something interpolated from their training set. In your example that's just a pollution of the training set by spam, but that's not that much of an issue in practice, as AI has been better than humans at classifying spam for over a decade no…

This is confusing to read If I agree with your definition of hallucinations in the context of LLMs... Then isn't your second paragraph literally just a way to artificially increase the likelihood of them occurring? You seem to differentiate between a hallucination caused by poisoning the dataset vs a hallucination caused by correct data, but can you honestly make such a distinction considering just how much data goes…

Yes, I can make such a distinction - if what the LLM is producing is in the training data then it's not a "hallucination". Note that this is an entirely separate problem from whether the LLM is "correct". In other words, I'm treating the LLM as a Chronicler, summarizing and reproducing what others have previously written, rather than as a Historian trying to determine the underlying truth of what occurred.

Re: 70% of new NPM packages in last 6 months were spam

#109

Post by 3np » Wed, Aug 07, 2024, 11:12 PM UTC

pdimitar wrote: post #73

lolinder wrote: post #66

Earlier quoted context omitted.

Which, incidentally, some people seem to have forgotten when suggesting that NPM should start deleting things en masse.

What other outcome than "curation" do you see as a solution of the "bloat" problem?

One possible outcome: Start considering it a public namespace (which in practice it's been not far from for a long time).

Meaning curation falls outside it and should not be centrally and unilaterally enforced by gating the entry.

We seem to be handling the bloated .com DNS namespace just fine.

Re: 70% of new NPM packages in last 6 months were spam

#110

Post by pdimitar » Wed, Aug 07, 2024, 11:30 PM UTC

3np wrote: post #109

pdimitar wrote: post #73

Earlier quoted context omitted.

What other outcome than "curation" do you see as a solution of the "bloat" problem?

One possible outcome: Start considering it a public namespace (which in practice it's been not far from for a long time). Meaning curation falls outside it and should not be centrally and unilaterally enforced by gating the entry. We seem to be handling the bloated .com DNS namespace just fine.

> We seem to be handling the bloated .com DNS namespace just fine.

If you say so. I wouldn't think that.