Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ride down into JavaScript dependency hell

blog.appsignal.com

Post reply on HN
Print view View original

111–120 of 149 posts

Re: Ride down into JavaScript dependency hell

#111

Post by hombre_fatal »

BiteCode_dev wrote: post #46
digianarchist wrote: post #39

Earlier quoted context omitted.

This is what happens with a wafer-thin standard library.

Yes and no. Because of the small stdlib, you need dependancies, sure. But why this idea that each dependency must be very small, forcing you to install a hundreds childs, which, themselves, will install a hundred until the pyramid of files start to turn node_modules into a stress test?

One idea that probably got encoded in the culture is that this code would also have to ship to the client. By having small deps, nobody had to wait around for more advanced tree-shaking (and its idiosyncrasies) to enter the picture.

Re: Ride down into JavaScript dependency hell

#112

Post by lmkg »

ng12 wrote: post #84
rootlocus wrote: post #81

Earlier quoted context omitted.

> - Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems. The post contains a graph with the package counts for other languages [1] 1 https://d33wubrfki0l68.cloudfront.net/7481900a584f733e7e9db7...

Is node's biggest crime is that it's easy to publish packages? Is this an argument for intentionally complicating package systems?

The issue with NPM is that packages tend to be more granular than other languages.

For example: The npm package "is-even" depends on the "is-odd" package. (In case it's not clear, this is a real package, not a made-up example.) In most other languages, these would be bundled together into an Integer Utilties package, if they're not already part of the standard library. In NPM, they are separate packages.

I believe there is, or was, a technical reason for NPM to have very fine-grained imports. Perhaps something about not supporting partial imports? I can't exactly find it. But the point is: NPM having more packages does not translate to a larger and more vibrant ecosystem. In some cases it simply means accomplishing the same thing requires a larger dependency graph.

That's not to say it's inherently a bad thing. But that's at least why it's not inherently a good thing either.

[1] https://www.npmjs.com/package/is-even

Re: Ride down into JavaScript dependency hell

#113

Post by [deleted] »

tpetry wrote: post #105
matheusmoreira wrote: post #55

Earlier quoted context omitted.

The problem is not small packages or huge dependency trees. It's the fact widely used packages have to be installed at all. This is compounded by the package manager which allows packages to have independent copies of their dependencies. We should have a de facto standard library that gets distributed along with the language's reference implementation. Widely-used packages that have a stable and well-designed interfa…

The problem is not small packages. The problem is there are many of these small libraries trying to solve the same problem (extending the small standard library more). And when there are no clear winners your dependency tree will be full of packages trying to do the same.

[deleted]

Re: Ride down into JavaScript dependency hell

#114

Post by dwohnitmok »

ng12 wrote: post #108
stickfigure wrote: post #95

Earlier quoted context omitted.

I'm totally baffled by this statement. Find me a mainstream language that's more complicated than this? "Checkout and build" works pretty much everywhere? I mean, you need the right version of mvn or pip or whatnot... just like you need the right version of npm. If anything, node/npm deserves special mention for NOT having reliable repeatable builds. Part of this is that a lot of packages rely on native code. But mos…

To put it simply to build a modern Java project you need the jdk + bazel + docker + maven + runtime classpath for a sub-set of npm's features (as far as I know you still can't include two versions of the same maven dependency). For a Node project you just need node. npm is part of node, and yarn is optional. The repeatable builds was a real issue but that's been solved years ago.

That's not true for Java. Most require Maven and then it's mvn install (you need the JDK for maven, but that feels similar to saying you need node for npm, they effectively come together).

Npm's dev dependencies are analogous to Maven's plugins.

Bazel is an alternative to maven. Docker fulfills the same role as in the Node ecosystem. The runtime class path is an implementation detail that you need to concern yourself with if you wish to run your Java code without Maven (analogous to running your JavaScript code without npm).

The two forms of the same dependency is a limitation that stems from Java the language rather than its ecosystem.

FWIW, nearly every language ecosystem I know of is at the point of "download build tool, run build tool."

Re: Ride down into JavaScript dependency hell

#115

Post by tln »

matheusmoreira wrote: post #69
tln wrote: post #66

Earlier quoted context omitted.

The package manager also has evolved to handle lots of dependencies. In the article 19000 packages are installed in 40 seconds... My Django project with 100 times fewer dependencies takes longer in CI Npm comes with an audit tool I have never seen dependency resolution fail (since packages can have private copies), unlike pypi or rubygems. So some of the downsides to a large dependency tree are mitigated. I'll add on…

> 19000 dependencies Doesn't this happen because of package manager duplication? I think npm lets packages have their own copies of their dependencies. Since we have a lots of small, widely used packages, they get duplicated at numerous points in the dependency graph. The number of files and installation size explodes.

A big part is the development toolchain, which would be installed at a system level for most languages.

I started a vue project last night, npm install --production installs 4 packages. With dev dependencies, I get 2300 packages. Eslint, babel, webpack, etc bring in lots of luggage

BTW, I think 19000 is wrong, on a fresh node_modules I get:

    $ npm install gatsby
    ...
    + gatsby@2.20.18
    added 1773 packages from 733 contributors 
     and audited 23706 packages in 52.317s
    $ du -sh node_modules
    245M node_modules
Not sure where the "audited" number comes from, but its not the number of install packages. I get 2737 directories containing a package.json, 1477 of which are uniq.

`debug` appears 32 times!

Re: Ride down into JavaScript dependency hell

#116

Post by erik_seaberg »

leetrout wrote: post #10
pizzapim wrote: post #4

So what does NPM do differently than plugin managers from other languages? I can't name one where the dependency hell is this big.

My personal take having “grown up” with all of this - JS is one of the worst ecosystems for relying on external packages for everything. It’s more the culture than the tooling. One of the things I love about Go is generally people are a little more forgiving to copying something around a couple times instead of making a lib for everything.

Copying code is pinning a dependency, committing to never update it, and hiding the evidence.

Re: Ride down into JavaScript dependency hell

#117

Post by coffeefirst »

seemslegit wrote: post #86
jessaustin wrote: post #82

Earlier quoted context omitted.

If these claims were true, we would have seen some examples by now. Instead, we get some tired invocations of "leftpad!" along with some more tired FUD. Who has been hacked because they used javascript instead of some other "more mature" language?

There is more to dependency risk than 'getting hacked' - correctness, maintainability, IP due diligence all play a role.

Yes. I've had several apps that can basically never be upgraded due to complete insanity deep in their dependency trees.

I'm pretty cautious about third party dependancies, but some of those libraries' libraries are not, and it's really hard to see that coming.

If you expect to live with code for 5+ years (and I understand that's not everyone, but it's almost always me), this is a big problem.

Re: Ride down into JavaScript dependency hell

#118

Post by mythrwy »

bob1029 wrote: post #97

Sometimes I feel like I am becoming increasingly out-of-touch with reality with how far this NPM/JS ecosystem nonsense has been taken. Is anyone still using this crap for de novo business applications that have real-world consequences? I have no problem if you want to build fantastical, sprawling node implementations for fun and potentially personal profit, but when your decisions start to involve other professionals…

Full page POST reloads on each data submission aren't going to fly with most users, even business users. Neither is reloading a page to get new data.

My personal feeling about CRUD is this, you don't have to get fully on the crazy train and spend all your time with packages and build tools to achieve a nice product. Yarn is better for me. Very sparing package use. And a little light Vue with server side templating. No I don't need you managing routes there JS framework, that never was a good idea in my estimation. (Or maybe it is and I don't get it but I like incremental changes and simplicity).

Once the dust settles perhaps I'll move on to a more complex setup but I don't have time for it now. I'm trying to get stuff done. However this doesn't have to mean eliminating modern behavior or appearance. Which means some JavaScript. It's going to be required in general by end users.

Re: Ride down into JavaScript dependency hell

#119

Post by jessaustin »

coffeefirst wrote: post #117
seemslegit wrote: post #86

Earlier quoted context omitted.

There is more to dependency risk than 'getting hacked' - correctness, maintainability, IP due diligence all play a role.

Yes. I've had several apps that can basically never be upgraded due to complete insanity deep in their dependency trees. I'm pretty cautious about third party dependancies, but some of those libraries' libraries are not, and it's really hard to see that coming. If you expect to live with code for 5+ years (and I understand that's not everyone, but it's almost always me), this is a big problem.

We've all lived with code that was 5+ years old, even if not all those years were ours. b^) Often that meant wrangling envars to point to the right ancient binaries, assuming we could somehow get those binaries installed. None of that is required with javascript. If a dependency is causing problems, just replace it with a different one. (npmjs.com has lots of packages!) Or, write something yourself. In five years you'll find enough spare time.

Re: Ride down into JavaScript dependency hell

#120

Post by BiteCode_dev »

hombre_fatal wrote: post #111
BiteCode_dev wrote: post #46

Earlier quoted context omitted.

Yes and no. Because of the small stdlib, you need dependancies, sure. But why this idea that each dependency must be very small, forcing you to install a hundreds childs, which, themselves, will install a hundred until the pyramid of files start to turn node_modules into a stress test?

One idea that probably got encoded in the culture is that this code would also have to ship to the client. By having small deps, nobody had to wait around for more advanced tree-shaking (and its idiosyncrasies) to enter the picture.

Make sense.
Post reply on HN