Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ride down into JavaScript dependency hell

blog.appsignal.com

Post reply on HN
Print view View original

41–50 of 149 posts

Re: Ride down into JavaScript dependency hell

#41

Post by tristanstcyr »

dmitriid wrote: post #2

I'm not buying the narrative of "js dependency hell what are they thinking" anymore. Try installing a package for Rust. Or Go. Or ... any language, really.

I agree that this isn't totally specific to JavaScript. It's just that the ecosystem is very extensive and so there are a lot of packages, because you don't need to write your own code for a lot of existing problems.

There is perhaps a something about the JavaScript audience tending to attract less advanced implementers. So you end up with a lot of otherwise considered trivial and low-quality utilities being published and pulled down from the registry.

Rust and Go tend to be for more advanced scenarios where things like performance, security and support matters tremendously.

Re: Ride down into JavaScript dependency hell

#42

Post by tluyben2 »

I cannot understand how people work with this. I work with many different technologies and try to avoid JS, but some times I have to. The past weeks I have worked on a React Native project that was written by someone else; what a horror show. I mean it wasn't the worst code by the previous guy but even in a few months a lot is simply broken and not 'best practice' anymore. Compared to most other environments I work with, iteration is somewhat faster when you finally have everything working, but the dependencies and ecosystem is horrible imho.

A bigger issue with all of it is that in my line of work, for the backend of firmware I cannot just depend on 3rd party libs as they will get audited, so I need to audit them before. Ofcourse most people don't have that issue.

Re: Ride down into JavaScript dependency hell

#43

Post by BiteCode_dev »

dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

Which is weird. I would have expected have the leftpad scandal that the community would have found a balance.

To me, culture is a feature, and despite using JS a lot since it has a monopoly on the browser, the community culture never clicked with me.

I often feel like they are reinventing the wheel, not learning from other techs solving the same problems, under and over-engineering, regularly guilty of the XKCD 927 syndrom.

Few projects taste integrated, working together, clean. The stacks looked like hacked together. Agreement on best practices and conventions are timidly showing up. It also seems popular tools are just starting to stabilize and the breaking modifications rhythm to slow down. At least more than anywhere else I worked.

There is no way I can be objective saying this, but anyone with the same feeling?

Re: Ride down into JavaScript dependency hell

#44

Post by oefrha »

dgellow wrote: post #11
awinter-py wrote: post #3

express (common web framework) seems to have only 51 deps in lockfile jekyll has 13 I don't disagree that lots of deps = supply chain risk But there have always been variations between projects & kinds of projects re how many deps they pull in. Try following someone's ipython data science tutorial, it's requirements.txt for days I think lack of a standard lib early on plus hipster functional culture made the '10s JS…

A python equivalent to "express" would be Flask https://palletsprojects.com/p/flask/ I see 4 dependencies in the setup.py, and they are all from the same team of maintainers (i.e the Pallet team): - Werkzeug - Jinja2 - itsdangerous - click https://github.com/pallets/flask/blob/master/setup.py https://github.com/pallets/flask/network/dependencies

To be fair, express is a production server itself whereas werkzeug is not, so you need to add something like gunicorn or uwsgi (yeah I know nginx supports uwsgi_pass natively). click is not necessary in production though (it's only used in the dev CLI).

That said, 4 dependencies is the norm in Python land, whereas 13 dependencies is an outlier in JS land.

Edit: oops, misread, gp said express has 51 deps, not 13.

Re: Ride down into JavaScript dependency hell

#45

Post by doubleunplussed »

jfkebwjsbx wrote: post #40
dgellow wrote: post #19

I think that we are in a situation where the number of dependency doesn't have any meaning anymore for people working on web frontend projects. Recently while revamping parts of the CI/CD at $WORK, I found out that running npm install for one of our frontend project downloads around 1 million dependencies. And npm audit reports a completely ridiculous number of security issues with them. It's just so absurd and nonse…

How is this even possible? How are there 1 million interesting packages?! In my job we use like, I don't know, 20 libraries total, out of, say, 50 alternatives that could do the job. Including dependencies of the dependencies I don't think we reach anywhere near 100. Every dependency is discussed with the entire team when we add it. How do you even have time to manage 1 million?!

Presumably almost all of the dependencies are transient dependencies, not direct ones.

I still can't fathom getting to a million though.

Re: Ride down into JavaScript dependency hell

#46

Post by BiteCode_dev »

digianarchist wrote: post #39
dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

This is what happens with a wafer-thin standard library.

Yes and no.

Because of the small stdlib, you need dependancies, sure.

But why this idea that each dependency must be very small, forcing you to install a hundreds childs, which, themselves, will install a hundred until the pyramid of files start to turn node_modules into a stress test?

Re: Ride down into JavaScript dependency hell

#47

Post by parhamn »

tristanstcyr wrote: post #37
parhamn wrote: post #28

What’s worse is that the whole installing multiple versions of the same package for different sub dependencies doesn’t make any sense when when the language supports singletons and module level variables with has no way of specifying which version to import. I was shocked this behavior exists when a site broke by upgrading a sub dependency. Turned out they both installed react so they used a different “creatContext”…

React would normally be a peerDependency and so you shouldn't run into this problem if dependencies are specified properly. If not, NPM module flattening should take care of it. Then worst case, you can also tell webpack to bundle a specific installation. There are some ways to solve these problems...

Sure (upvote) and we dealt with it accordingly after some confused digging. On a more general level I’ve never seen something like that in a package manager. It seems as though some of the language features (like the singleton I mentioned) would make it obvious that arbitrarily duplicating packages and scoping them to sub packages is a major no-no, as in most languages I’ve dealt with.

Re: Ride down into JavaScript dependency hell

#48

Post by matheusmoreira »

dwheeler wrote: post #13

Part of the issue is that JavaScript packages are often far far smaller than packages in other ecosystems. "small packages are extremely common in the JavaScript npm package system. For example, in npm, 47% of the packages have 0 or 1 functions, and the average npm package has 112 physical lines of code. In contrast, the average Python module in the PyPI repository has 2,232 physical lines of code." Source: "Vulnerab…

Why are small packages bad? Independent functions should be versioned and distributed independently. Otherwise we get several utility packages which are nothing but collections of independent functions.

Widely used packages should form the basis of a standard library that is distributed with the language itself.

Re: Ride down into JavaScript dependency hell

#49

Post by renke1 »

crazygringo wrote: post #35
dgellow wrote: post #19

I think that we are in a situation where the number of dependency doesn't have any meaning anymore for people working on web frontend projects. Recently while revamping parts of the CI/CD at $WORK, I found out that running npm install for one of our frontend project downloads around 1 million dependencies. And npm audit reports a completely ridiculous number of security issues with them. It's just so absurd and nonse…

I mean npm only has slightly more than 1 million packages, last I checked. This doesn't make even the slightest bit of sense. Did somebody somehow just import... everything by accident? Did npm hit a bug? Is there some infinite recursion going on that eventually hits a wall? This obviously isn't representative.

1 million packages most of which have more than a single version.

Re: Ride down into JavaScript dependency hell

#50

Post by BiteCode_dev »

jfkebwjsbx wrote: post #40
dgellow wrote: post #19

I think that we are in a situation where the number of dependency doesn't have any meaning anymore for people working on web frontend projects. Recently while revamping parts of the CI/CD at $WORK, I found out that running npm install for one of our frontend project downloads around 1 million dependencies. And npm audit reports a completely ridiculous number of security issues with them. It's just so absurd and nonse…

How is this even possible? How are there 1 million interesting packages?! In my job we use like, I don't know, 20 libraries total, out of, say, 50 alternatives that could do the job. Including dependencies of the dependencies I don't think we reach anywhere near 100. Every dependency is discussed with the entire team when we add it. How do you even have time to manage 1 million?!

To quote the article:

"Luckily, most of them are using the same version of lodash, which just needs one lodash to install inside node_modules. This is often not the case with real-world production projects. Sometimes, different packages require different versions of other packages."

Npm lets you install several versions of the same package. This has the advantage of saving the dev the testing of their lib with a wide range of dependencies versions: it almost always works. Unfortunately, that means most devs just choose one version and call it a day. They build libs like one would build a project: as you were alone in the world.

Post reply on HN