Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ride down into JavaScript dependency hell

blog.appsignal.com

Post reply on HN
Print view View original

51–60 of 149 posts

Re: Ride down into JavaScript dependency hell

#51

Post by jfkebwjsbx »

sippeangelo wrote: post #9
bsagdiyev wrote: post #6

Earlier quoted context omitted.

I’m not sure whataboutism is the way to fix the well-known issues the JS ecosystem has in package management. When I’ve installed a Go package or Python package I haven’t ended up with sometimes hundreds or thousands of sub-dependencies. Some packages can be ridiculous, but nothing like I have seen running “npm i” for something that seems like it should be simple. I apologize for not having an example off-hand but th…

Try Feathers, a "lightweight web-framework for creating real-time applications and REST APIs" clocking in at about 600 transitive dependencies. How do you audit this?

I think they just don't audit nor review the code they depend on.

Re: Ride down into JavaScript dependency hell

#52

Post by jfkebwjsbx »

dmitriid wrote: post #24
bsagdiyev wrote: post #6

Earlier quoted context omitted.

I’m not sure whataboutism is the way to fix the well-known issues the JS ecosystem has in package management. When I’ve installed a Go package or Python package I haven’t ended up with sometimes hundreds or thousands of sub-dependencies. Some packages can be ridiculous, but nothing like I have seen running “npm i” for something that seems like it should be simple. I apologize for not having an example off-hand but th…

It's now whataboutism. It's finger pointing from a lot of people who develop in languages whose story is only marginally better. As I wrote in a sibling comment, I have a project I haven't even started yet in Rust [1]. And it's measly 6 dependencies pull a total of 197. [1] https://news.ycombinator.com/item?id=22841742

If what you say is true, then Rust is suffering the same fate.

197 dependencies is too many to trust.

Re: Ride down into JavaScript dependency hell

#53

Post by dragonwriter »

matheusmoreira wrote: post #48
dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

Why are small packages bad? Independent functions should be versioned and distributed independently. Otherwise we get several utility packages which are nothing but collections of independent functions. Widely used packages should form the basis of a standard library that is distributed with the language itself.

> Why are small packages bad?

Small, separately maintained packages are bad in a an ecosystem where a final system can incorporate only one version of any given package because it increases the number of opportunities for version conflicts.

Now, it's true that there are some other concerns which weigh in favor of packages being at the minimum useful size, but those necessarily also weigh in favor of a package management system which allows each package to isolate its upstream dependencies without constraining it's downstream users. And not just allows, but facilitates it so that it is the norm, so that dependency conflicts aren't a thing.

Re: Ride down into JavaScript dependency hell

#54

Post by williamdclt »

tluyben2 wrote: post #42

I cannot understand how people work with this. I work with many different technologies and try to avoid JS, but some times I have to. The past weeks I have worked on a React Native project that was written by someone else; what a horror show. I mean it wasn't the worst code by the previous guy but even in a few months a lot is simply broken and not 'best practice' anymore. Compared to most other environments I work w…

I don't agree with you on this. The JS (or NPM) way makes it a pain to audit a project due to thousands of dependencies, but it's far easier to "have everything working".

`npm i && npm start` is as good as it gets in terms of "getting started" friction (with version locking): python is worse, ruby is worse, PHP is worse, Java is worse.

React-Native has more moving pieces indeed, but it's not really JS's (or NPM's) fault: you get all problems of iOS and Android at once. I will definitely agree with you that working on React-Native is a pain, to be able to work with modern tools need to know JS, optionally TS, Obj-C, Swift, Java, Kotlin, Ruby (for Fastlane) and Gradle's custom language. Insane.

Re: Ride down into JavaScript dependency hell

#55

Post by matheusmoreira »

BiteCode_dev wrote: post #46
digianarchist wrote: post #39

Earlier quoted context omitted.

This is what happens with a wafer-thin standard library.

Yes and no. Because of the small stdlib, you need dependancies, sure. But why this idea that each dependency must be very small, forcing you to install a hundreds childs, which, themselves, will install a hundred until the pyramid of files start to turn node_modules into a stress test?

The problem is not small packages or huge dependency trees. It's the fact widely used packages have to be installed at all. This is compounded by the package manager which allows packages to have independent copies of their dependencies.

We should have a de facto standard library that gets distributed along with the language's reference implementation. Widely-used packages that have a stable and well-designed interfaces should be distributed by default. If something isn't being used anymore, it can be dropped from the standard library while still being available from the repositories.

Re: Ride down into JavaScript dependency hell

#56

Post by mschuetz »

matheusmoreira wrote: post #48
dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

Why are small packages bad? Independent functions should be versioned and distributed independently. Otherwise we get several utility packages which are nothing but collections of independent functions. Widely used packages should form the basis of a standard library that is distributed with the language itself.

Every separate package adds overhead and another maintainer that you've got to put your trust in. I'd rather have few packages of well trusted maintainers instead of a thousand packages from god knows who.

Personally, I've made it a habit to not add anything that requires trivial one liner garbage packages, which amounts to not installing any dependency that uses anything from Jon Schlinkert (so no Webpack). Unfortunately I'm stuck with gulp right now but with the next major rewrite, I'll also get rid of gulp and its 300 dependencies.

Re: Ride down into JavaScript dependency hell

#57

Post by mrighele »

shakna wrote: post #8
dmitriid wrote: post #2

I'm not buying the narrative of "js dependency hell what are they thinking" anymore. Try installing a package for Rust. Or Go. Or ... any language, really.

Gatsby has 19k dependencies. I don't think I can find something like that for Rust. The largest, and not exactly admirable, I can find is reqwest that drags in 97. Whilst 100 is a huge number... It's an enormous gap from the many thousands.

And that it is not even the worst I have seen. I think the worst offenders are development tools like webpack.

I have a small project that I work on from time to time which uses 5 libraries (react, a map library and a chart library), typescript, and react-scripts (which I guess pulls in webpack and all the rest).

This is what happens when I run a npm audit (I havent' touched it for a couple of months)

> found 38934 vulnerabilities (38916 low, 18 moderate) in 906346 scanned packages

> run `npm audit fix` to fix 38620 of them.

> 314 vulnerabilities require manual review. See the full report for details.

While the real numbers are probably lower because there is a lot of duplication inside the node_modules folder, I find this ... astounding.

Re: Ride down into JavaScript dependency hell

#58

Post by williamdclt »

brobdingnagians wrote: post #33
samsquire wrote: post #20

Earlier quoted context omitted.

My stuff breaks whenever I try upgrade it. People refactoring stuff.

These types of issues are the biggest reason why I avoid javascript/npm projects. I came back to a project after a few months and it was broken, had to rewrite some parts and upgrade other parts just to get it to run again.

I'm sorry but for all the flaws of the NPM ecosystem, version locking is not one (both `npm` and `yarn` do much much better than anything in Python). There's no reason a correctly set-up project would magically break with time.

I don't love NPM anymore than the next guy, but my blame will go to the dev in this case.

Re: Ride down into JavaScript dependency hell

#59

Post by matheusmoreira »

mschuetz wrote: post #56
matheusmoreira wrote: post #48

Earlier quoted context omitted.

Why are small packages bad? Independent functions should be versioned and distributed independently. Otherwise we get several utility packages which are nothing but collections of independent functions. Widely used packages should form the basis of a standard library that is distributed with the language itself.

Every separate package adds overhead and another maintainer that you've got to put your trust in. I'd rather have few packages of well trusted maintainers instead of a thousand packages from god knows who. Personally, I've made it a habit to not add anything that requires trivial one liner garbage packages, which amounts to not installing any dependency that uses anything from Jon Schlinkert (so no Webpack). Unfortun…

> I'd rather have few packages of well trusted maintainers instead of a thousand packages from god knows who.

Then the problem is the fact anybody can submit a package, not small packages.

Linux distributions solve this problem by having dedicated maintainers. Users of a distribution trust its maintainers when they use it. Software developers want the complete opposite: language-specific packages, instant and unrestricted package publication, containers, etc. Nobody wants to have to talk to a maintainer in order to get their software included in a distribution. Of course the result ends up being a mess.

What if Node's maintainers decided to distribute a curated set of packages alongside Node itself? The size of each individual package wouldn't really matter.

Re: Ride down into JavaScript dependency hell

#60

Post by dgellow »

williamdclt wrote: post #54
tluyben2 wrote: post #42

I cannot understand how people work with this. I work with many different technologies and try to avoid JS, but some times I have to. The past weeks I have worked on a React Native project that was written by someone else; what a horror show. I mean it wasn't the worst code by the previous guy but even in a few months a lot is simply broken and not 'best practice' anymore. Compared to most other environments I work w…

I don't agree with you on this. The JS (or NPM) way makes it a pain to audit a project due to thousands of dependencies, but it's far easier to "have everything working". `npm i && npm start` is as good as it gets in terms of "getting started" friction (with version locking): python is worse, ruby is worse, PHP is worse, Java is worse. React-Native has more moving pieces indeed, but it's not really JS's (or NPM's) fa…

I've done 3 years of Golang as my main language, my experience is:

- go build ./... builds everything from the current project

- go test ./... tests everything from the current project

- go install ./... installs everything from the current project

That's one of the best thing ever. I can go to any project, I know how to build, test, install, and can directly be productive.

On the other hand, I started learning C++ one year ago, and it's the exact opposite, each project has its own homemade build system that isn't supported by one platform or the other.

Post reply on HN