Ride down into JavaScript dependency hell • Hacker News Board

Re: Ride down into JavaScript dependency hell

#31

Post by thomasfoster96 » Sat, Apr 11, 2020, 2:16 PM UTC

pizzapim wrote: post #4

So what does NPM do differently than plugin managers from other languages? I can't name one where the dependency hell is this big.

I’d say compared to other languages there’s been two big issues:

1. A not insignificant number of packages are either polyfilling things in browsers, or providing a consistent (or ‘isomorphic') API for certain things across both browsers and Node.js, or adding things that should have been in a standard library.

2. A lot of packages still seem to be distributed primarily as CommonJS and not ES Modules. CommonJS makes tree-shaking harder than it should, so it was often just easier to break what should have been a single library into many smaller pieces.

Hopefully in the not too distant future library maintainers get around to reading the Node.js docs[0] fix some of this.

[0] https://nodejs.org/api/esm.html

Re: Ride down into JavaScript dependency hell

#32

Post by bsagdiyev » Sat, Apr 11, 2020, 2:19 PM UTC

dmitriid wrote: post #24

bsagdiyev wrote: post #6

Earlier quoted context omitted.

I’m not sure whataboutism is the way to fix the well-known issues the JS ecosystem has in package management. When I’ve installed a Go package or Python package I haven’t ended up with sometimes hundreds or thousands of sub-dependencies. Some packages can be ridiculous, but nothing like I have seen running “npm i” for something that seems like it should be simple. I apologize for not having an example off-hand but th…

It's now whataboutism. It's finger pointing from a lot of people who develop in languages whose story is only marginally better. As I wrote in a sibling comment, I have a project I haven't even started yet in Rust [1]. And it's measly 6 dependencies pull a total of 197. [1] https://news.ycombinator.com/item?id=22841742

I took a look, this sounds more unique to Rust than Go or Python, but they all can do it to some degree. I have no hands on experience with Rust to really have any reasonable input on this so I don’t want to make an uninformed statement.

Re: Ride down into JavaScript dependency hell

#33

Post by brobdingnagians » Sat, Apr 11, 2020, 2:21 PM UTC

samsquire wrote: post #20

Waterluvian wrote: post #16

I have a project with a total of eight dependencies. I walked away for six months as it was stable. I come back to add a few features and npm tells me I have over 38 000 vulnerabilities of different severity levels. So many that 'npm audit' just freezes up. So there's that too.

My stuff breaks whenever I try upgrade it. People refactoring stuff.

These types of issues are the biggest reason why I avoid javascript/npm projects. I came back to a project after a few months and it was broken, had to rewrite some parts and upgrade other parts just to get it to run again.

Re: Ride down into JavaScript dependency hell

#34

Post by johannes1234321 » Sat, Apr 11, 2020, 2:26 PM UTC

My concern with npm packages isn't the number and small size of the packages itself, but that most of them are maintained and owned by individuals who do work with little review by others and who could disappear any time. If an individual goes away or takes harmful action it takes time till this is noticed and till a surviving fork emerges.

If more libraries were maintained by groups in a shared/collaborative way, more of those risks would go away.

Re: Ride down into JavaScript dependency hell

#35

Post by crazygringo » Sat, Apr 11, 2020, 2:33 PM UTC

dgellow wrote: post #19

I think that we are in a situation where the number of dependency doesn't have any meaning anymore for people working on web frontend projects. Recently while revamping parts of the CI/CD at $WORK, I found out that running npm install for one of our frontend project downloads around 1 million dependencies. And npm audit reports a completely ridiculous number of security issues with them. It's just so absurd and nonse…

I mean npm only has slightly more than 1 million packages, last I checked.

This doesn't make even the slightest bit of sense. Did somebody somehow just import... everything by accident? Did npm hit a bug? Is there some infinite recursion going on that eventually hits a wall?

This obviously isn't representative.

Re: Ride down into JavaScript dependency hell

#36

Post by dep_b » Sat, Apr 11, 2020, 2:34 PM UTC

dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

iOS dependencies in Cocoapods are including less than 1 other packages on average (1). The most I see are Alamofire (a networking package people keep using for reasons that I do not fully understand) and Starscream (a Websocket implementation, that's pretty hard to write from scratch to be honest). Having unnecessary dependencies is pretty much frowned upon.

One of the reasons is that the iOS SDK is really complete by itself while Javascript doesn't even ship with a usable Date implementation. If somebody could just come up with a standard library for JavaScript that is really widely adopted I think the amount of packages can be cut back dramatically.

(1) totally based on gut feeling, not based on hard data

Re: Ride down into JavaScript dependency hell

#37

Post by tristanstcyr » Sat, Apr 11, 2020, 2:34 PM UTC

parhamn wrote: post #28

What’s worse is that the whole installing multiple versions of the same package for different sub dependencies doesn’t make any sense when when the language supports singletons and module level variables with has no way of specifying which version to import. I was shocked this behavior exists when a site broke by upgrading a sub dependency. Turned out they both installed react so they used a different “creatContext”…

React would normally be a peerDependency and so you shouldn't run into this problem if dependencies are specified properly.

If not, NPM module flattening should take care of it.

Then worst case, you can also tell webpack to bundle a specific installation.

There are some ways to solve these problems...

Re: Ride down into JavaScript dependency hell

#38

Post by hn_throwaway_99 » Sat, Apr 11, 2020, 2:40 PM UTC

dep_b wrote: post #36

dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

iOS dependencies in Cocoapods are including less than 1 other packages on average (1). The most I see are Alamofire (a networking package people keep using for reasons that I do not fully understand) and Starscream (a Websocket implementation, that's pretty hard to write from scratch to be honest). Having unnecessary dependencies is pretty much frowned upon. One of the reasons is that the iOS SDK is really complete b…

Java didn't have a decent date implementation until Java 8.

> If somebody could just come up with a standard library for JavaScript that is really widely adopted I think the amount of packages can be cut back dramatically.

As this article points out, I think that lodash is that for many people.

Re: Ride down into JavaScript dependency hell

#39

Post by digianarchist » Sat, Apr 11, 2020, 2:40 PM UTC

dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

This is what happens with a wafer-thin standard library.

Re: Ride down into JavaScript dependency hell

#40

Post by jfkebwjsbx » Sat, Apr 11, 2020, 2:41 PM UTC

dgellow wrote: post #19

I think that we are in a situation where the number of dependency doesn't have any meaning anymore for people working on web frontend projects. Recently while revamping parts of the CI/CD at $WORK, I found out that running npm install for one of our frontend project downloads around 1 million dependencies. And npm audit reports a completely ridiculous number of security issues with them. It's just so absurd and nonse…

How is this even possible?

How are there 1 million interesting packages?!

In my job we use like, I don't know, 20 libraries total, out of, say, 50 alternatives that could do the job. Including dependencies of the dependencies I don't think we reach anywhere near 100. Every dependency is discussed with the entire team when we add it.

How do you even have time to manage 1 million?!