Live data from Hacker News

Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

rust-lang.org

121–130 of 307 posts

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#121
post #44

Earlier quoted context omitted.

> This entire article is a pretty damning report on JavaScript in general How so?

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

The k8s team probably should, judging from the state of their codebase.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#122
post #50

Earlier quoted context omitted.

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

> Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). Why do people want this? The builds are no longer reproducible, security and edge case issues can come out of nowhere, api changes from an irresponsible maintainer can break things, network and resource failure can break the build, it's just a terrible idea. The proper use of a semvar system is entir…

Reproducibility was a core concern for cargo. Your parent is incorrect. A lock file means that your dependencies are never updated unless you explicitly ask for an update.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#123
post #101

Earlier quoted context omitted.

How much is that actually necessary these days? I remember spending ages tweaking flags in the 1.4 - 1.6 years. But there have increasingly been sensible defaults with broad applicability. Now that G1 is both default and usable, even more so. As i've moved onto 11, i've slashed our apps' JAVA_OPTS down to almost nothing - max heap size, some GC logging flags, that's it.

At scale? Very necessary. The default GC as of Java 8 (the last version I used in production) suffered significant performance issues above 40 GB of working set, and required careful tuning thereafter.

I completely agree. The JVM is fine until it isn't, and at that point it becomes very frustrating to deal with - even when using less than 16GB RAM.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#124
post #72

Earlier quoted context omitted.

Not so ironic—just because a practice is common among the median engineer in an ecosystem, doesn’t mean it will be common among the most experienced engineers in that ecosystem. Node gets a bad rap mostly for the fact that it has tons and tons of inexperienced engineers using it (probably as one of their first programming languages.) Same reason PHP got a bad rap back when. You can build solid software in both, by fi…

We're not talking about average JS developers, we're talking about the NPM maintainers. EDIT: I did miss that. Sorry for the noise.

You missed the joke. Let me quote the GP again:

> I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one.

For this to be "ironic", having lots and lots of dependencies would actually have to be a Node.js "best practice." But it's not.

It seems like a best practice to outsiders, for the same reason that setting `register_globals` seemed like a "best practice" in PHP back in PHP3/4. Because it was extremely common, one might assume that it was endorsed as a canonical approach. And so you do it yourself, and write sophomoric tutorials suggesting others do the same, perpetuating the problem.

In reality, the "best practice" followed by experienced software engineers (for Node.js or any other language) is to carefully consider your dependencies, and to try to avoid dependencies that cause an explosion of sub-dependencies. The NPM maintainers are experienced engineers, and so they follow this best practice.

There is no irony here. It is not "the Node.js way" to use tons and tons of dependencies, such that the NPM maintainers are going against the grain somehow. It's just the way of programmers inexperienced in engineering to not care about dependency proliferation; and then, further, to make a large set of their own tiny libraries (with already-exploded deps trees) because they aren't yet at a stage of programming expertise where they see that code as trivial to bang out whenever they need it (see: the left-pad package) that then further encourages others to depend on them. It's the "copying and pasting a solution together from bad code in five StackOverflow posts" phase of one's programming career, except instead of having to copy-and-paste, all the snippets are symbolically linked together into a big tree and you refer to them by name. (Again, that's not an indictment of Node.js—there's nothing you can do to stop a bunch of inexperienced engineers from doing this to your package ecosystem as well, if they happen to be drawn into your community.)

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#125
post #44

Earlier quoted context omitted.

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

The k8s team probably should, judging from the state of their codebase.

I’m interested to see what becomes of https://www.cloudatomiclab.com/rustyk8s/

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#126

Hey folks! This is part of our whitepaper series. This means that the audience is CTOs of larger organizations, and so the tone and content are geared for that, more than HN. Please keep that in mind!

Are CTO's of largish organizations still not part of the hacker news audience ...? That seems a bit of a damning statement to be about the population of CTO's ...

I'm sure some are on HN, but it's definitely not "the place" for them, and many have probably never heard of it

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#127
post #5

Title could be improved, as I was wondering if the command line tool npm had itself been rewritten. It’s actually one of npm’s web services that was rewritten. FTA: “Java was excluded from consideration because of the requirement of deploying the JVM and associated libraries along with any program to their production servers. This was an amount of operational complexity and resource overhead that was as undesirable a…

I'm not a fan of relying on distro package managers for installation of runtime dependencies on servers. Too many opportunites for variables to creep in if the version isn't locked, and then having to make sure all the package manager dependencies and config themselves. Even if you automate you're still at the mercy of the repo to have the version you need etc. and often times you need to customize the install for a…

What's the alternative? Not installing the dependencies so the app doesn't work?

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#128

Earlier quoted context omitted.

Well, speaking from experience with a JBoss application layer in a recent software project I worked on: New java versions do break existing libraries or apps, and need to be tested thoroughly. When the company hasn't budgeted for that expense, it becomes difficult to update. Often an architect or software team will insist on using the Oracle JVM rather than the included openjvm. That adds extra steps to download, sto…

> New java versions do break existing libraries or apps Have you used the Rust compiler? My experience tells me that any Github Rust project not updated in the last two years doesn't work with my Rust compiler. Whereas Java apps written a decade ago still compile and run on OpenJDK/Oracle often with zero or near zero changes. > Often an architect or software team will insist on using the Oracle JVM rather than the in…

> any Github Rust project not updated in the last two years doesn't work with the latest Rust compiler.

I don't think this is correct? Rust introduced an "editions"-based system in 2015, with the very aim of ensuring forward compatibility on a crate-by-crate basis. The aim is definitely that any crate written to be compatible with some "edition of Rust", whether 2015, 2018 or whatever, can be made to compile on a future version of the compiler.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#129
post #44

Earlier quoted context omitted.

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

The k8s team probably should, judging from the state of their codebase.

If you feel that way, this is pretty funny then: https://news.ycombinator.com/item?id=19295841

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#130
post #5

Title could be improved, as I was wondering if the command line tool npm had itself been rewritten. It’s actually one of npm’s web services that was rewritten. FTA: “Java was excluded from consideration because of the requirement of deploying the JVM and associated libraries along with any program to their production servers. This was an amount of operational complexity and resource overhead that was as undesirable a…

Deploying jvm app/service is now even more complex than figuring out which version of node to use to run a service. Is it supposed to be oracle java? Openjdk? Adoptopenjdk or one of a half dozen more? Which version of it? Anything to tweak in gc or startup settings for it/the version? Do we need to regression test the service on minor jdk upgrade? Is that jvm compatible with some os version we are running that has so…

> Deploying jvm app/service is now even more complex than figuring out which version of node to use to run a service. > Is it supposed to be oracle java? Openjdk? Adoptopenjdk or one of a half dozen more? Which version of it?

All of these must have been tested against the Java TCK. So, the answer is "whichever you/your company decided is okay" for the first part and for the second part "newest update of the Java version your software runs on". Doesn't sound very complex to me.

Post reply on HN