Ride down into JavaScript dependency hell • Hacker News Board

Re: Ride down into JavaScript dependency hell

#71

Post by dep_b » Sat, Apr 11, 2020, 4:18 PM UTC

hn_throwaway_99 wrote: post #38

dep_b wrote: post #36

Earlier quoted context omitted.

iOS dependencies in Cocoapods are including less than 1 other packages on average (1). The most I see are Alamofire (a networking package people keep using for reasons that I do not fully understand) and Starscream (a Websocket implementation, that's pretty hard to write from scratch to be honest). Having unnecessary dependencies is pretty much frowned upon. One of the reasons is that the iOS SDK is really complete b…

Java didn't have a decent date implementation until Java 8. > If somebody could just come up with a standard library for JavaScript that is really widely adopted I think the amount of packages can be cut back dramatically. As this article points out, I think that lodash is that for many people.

From what I read about it it’s more like syntactic sugar, not something alike the .Net core libraries.

Get your JS Core libraries once from a repo, reuse them in every application that uses the same ones

Re: Ride down into JavaScript dependency hell

#72

Post by matheusmoreira » Sat, Apr 11, 2020, 4:18 PM UTC

oefrha wrote: post #65

matheusmoreira wrote: post #59

Earlier quoted context omitted.

> I'd rather have few packages of well trusted maintainers instead of a thousand packages from god knows who. Then the problem is the fact anybody can submit a package, not small packages. Linux distributions solve this problem by having dedicated maintainers. Users of a distribution trust its maintainers when they use it. Software developers want the complete opposite: language-specific packages, instant and unrestr…

> What if Node's maintainers decided to distribute a curated set of packages alongside Node itself? First, front-end has nothing to do with Node. Secondly, let’s say there’s some sort of front-end foundation who decides to provide a curated set of packages. A natural candidate would be create-react-app, which pulls over a thousand dependencies. That’s just one package you want to include. How the hell do you even sta…

> First, front-end has nothing to do with Node.

It was just an example.

> A natural candidate would be create-react-app, which pulls over a thousand dependencies. That’s just one package you want to include. How the hell do you even start to curate such a beast?

I don't know. How did Linux distributions do it? Arch Linux has over 10 thousand packages. Debian has over 50 thousand packages. Maybe people should ask the maintainers.

Re: Ride down into JavaScript dependency hell

#73

Post by oefrha » Sat, Apr 11, 2020, 4:23 PM UTC

matheusmoreira wrote: post #72

oefrha wrote: post #65

Earlier quoted context omitted.

> What if Node's maintainers decided to distribute a curated set of packages alongside Node itself? First, front-end has nothing to do with Node. Secondly, let’s say there’s some sort of front-end foundation who decides to provide a curated set of packages. A natural candidate would be create-react-app, which pulls over a thousand dependencies. That’s just one package you want to include. How the hell do you even sta…

> First, front-end has nothing to do with Node. It was just an example. > A natural candidate would be create-react-app, which pulls over a thousand dependencies. That’s just one package you want to include. How the hell do you even start to curate such a beast? I don't know. How did Linux distributions do it? Arch Linux has over 10 thousand packages. Debian has over 50 thousand packages. Maybe people should ask the…

Point to me one package in Debian that depends on thousands more.

Oh and I was a maintainer for a major package manager back in the day for crying out loud (was responsible for overseeing the general health of the project as well as directly responsible for maybe a couple dozen individual packages). Never seen this kind of madness.

Re: Ride down into JavaScript dependency hell

#74

Post by matheusmoreira » Sat, Apr 11, 2020, 4:44 PM UTC

oefrha wrote: post #73

matheusmoreira wrote: post #72

Earlier quoted context omitted.

> First, front-end has nothing to do with Node. It was just an example. > A natural candidate would be create-react-app, which pulls over a thousand dependencies. That’s just one package you want to include. How the hell do you even start to curate such a beast? I don't know. How did Linux distributions do it? Arch Linux has over 10 thousand packages. Debian has over 50 thousand packages. Maybe people should ask the…

Point to me one package in Debian that depends on thousands more. Oh and I was a maintainer for a major package manager back in the day for crying out loud (was responsible for overseeing the general health of the project as well as directly responsible for maybe a couple dozen individual packages). Never seen this kind of madness.

I don't know about Debian but Arch Linux has huge package groups and metapackages.

https://www.archlinux.org/groups/x86_64/kde-applications/

https://www.archlinux.org/groups/x86_64/pro-audio/

The Arch Wiki recommends the use of these huge packages. I don't see any reason why it wouldn't scale to a thousand packages or more.

Re: Ride down into JavaScript dependency hell

#75

Post by fxtentacle » Sat, Apr 11, 2020, 4:45 PM UTC

overgard wrote: post #68

Sadly the alternative isn't much better: the old school model of not having packages often leads to libraries never getting updated at all. In C++ often adding a library is such a chore once you get through all the linking issues, as you battle the 4000 conflicting compiler switches, and compiler versions, that once it works you never want to change it again. I think maybe the best solution we have so far are robust…

I feel like Bazel and Cmake have solved this sufficiently well. I regularly compile larger projects from scratch and usually the process is to call vcvars (I'm on Windows) and then Bazel and then an hour later I have a folder full of cached lib and dll files so that linking my actual executable only takes a second.

I would be so incredibly grateful to have something comparable in JavaScript that quickly checks static typing for my entire project. But the closest I have found is Rust/Elm, so not using JavaScript anymore.

Re: Ride down into JavaScript dependency hell

#76

Post by tmcw » Sat, Apr 11, 2020, 4:53 PM UTC

How to write a silly post about JavaScript dependencies:

- Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems.

- Talk about devDependencies and dependencies without considering how they might be entirely different, say… one kind gets bundled, the other doesn't.

- Always use npm, so you can highlight duplicates. Don't use yarn.

- Adopt a narrow definition of 'dependency hell' so you can focus on the thing that JavaScript does imperfectly (minimizing number of dependencies) and avoid talking about the things that it does well (handling conflicting deps, solving diamond deps, resolving dependency chains)

- Don't worry about explaining how any of the issues you're discussing have real-world implications. Have companies been failing because they run out of disk space because of node_modules?

- Take automated security audits extremely seriously. A backtracking bug in the argument parser for the web framework's local CLI tool? Of course it's not exploitable, when… it's a web app, and the CLI tool is something you run locally. But add it to the count as if it is!

Re: Ride down into JavaScript dependency hell

#77

Post by fxtentacle » Sat, Apr 11, 2020, 4:55 PM UTC

pizzapim wrote: post #4

So what does NPM do differently than plugin managers from other languages? I can't name one where the dependency hell is this big.

It's the users. And money.

JavaScript is super popular and used by companies so there are lots of people releasing their own open source project in an attempt to hustle into a well-paid job by looking more experienced than they really are.

I'd say it's the same reason why there are so many bad Java and Php tutorials.

C++ for comparison is shielded from this because it's not the kind of programming language that you would learn in a weekend seminar to improve your salary.

Re: Ride down into JavaScript dependency hell

#78

Post by tomaszs » Sat, Apr 11, 2020, 5:04 PM UTC

I treat Npm packages like a JavaScript and PHP. There was a need and it was done. In the short run it is good enough. But in the long run it may become complicated. For example what about essencial packages that wont be maintained anymore? What about depending on millions of lines of code from unknown sources? What about single dependency that is npm? It is a risk too.

Re: Ride down into JavaScript dependency hell

#79

Post by seemslegit » Sat, Apr 11, 2020, 5:07 PM UTC

tmcw wrote: post #76

How to write a silly post about JavaScript dependencies: - Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems. - Talk about devDependencies and dependencies without considering how they might be entirely different, say… one kind gets bundled, the other doesn't. - Always use npm, so you can highlight duplicates. Don't use yarn. - Adopt a narrow definition of 'depend…

How to downplay the severity of the problem:

Ignore the ridiculously small functionality scope that acceptably qualifies for a js package in a way that does not exist in any other ecosystem and ads a big multiplier in front of all the other risks involved.

Re: Ride down into JavaScript dependency hell

#80

Post by jessaustin » Sat, Apr 11, 2020, 5:08 PM UTC

parhamn wrote: post #47

tristanstcyr wrote: post #37

Earlier quoted context omitted.

React would normally be a peerDependency and so you shouldn't run into this problem if dependencies are specified properly. If not, NPM module flattening should take care of it. Then worst case, you can also tell webpack to bundle a specific installation. There are some ways to solve these problems...

Sure (upvote) and we dealt with it accordingly after some confused digging. On a more general level I’ve never seen something like that in a package manager. It seems as though some of the language features (like the singleton I mentioned) would make it obvious that arbitrarily duplicating packages and scoping them to sub packages is a major no-no, as in most languages I’ve dealt with.

It seems like a weird thing about React, or maybe how you're using it, that this is a problem. Most packages, and the language itself, handle multiple simultaneous versions of dependencies without issue. If packages are doing the old-fashioned "stick a property on window" thing instead of just getting required or imported browserify-style, they can step on each others' toes (and it's totally possible that React is dumb enough to step on its own toes). It has been years since that was necessary. browserify or more recent tools like parcel or webpack should be used instead.