Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

rust-lang.org

Post reply on HN
Print view View original

41–50 of 307 posts

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#41

Post by Ygg2 »

mlthoughts2018 wrote: post #18

> ““Java was excluded from consideration because of the requirement of deploying the JVM and associated libraries along with any program to their production servers. This was an amount of operational complexity and resource overhead that was as undesirable as the unsafety of C or C++.” To me this reads like someone has a chip on their shoulder to demand that Rust be viewed as generally superior instead of merely bein…

> It’s the same with C or C++. These languages support a type of low-level memory and data model that is permissive in ways that can lead to errors if not used properly.

To be honest, I don't think I know of single piece of C/C++ code that doesn't suffer from security exploits from "improper" memory use.

Chrome does, Firefox does, Linux does. Hell car and pacemaker software suffers from exploits, most likely caused by improper memory handling.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#42

Post by oweiler »

javagram wrote: post #5

Title could be improved, as I was wondering if the command line tool npm had itself been rewritten. It’s actually one of npm’s web services that was rewritten. FTA: “Java was excluded from consideration because of the requirement of deploying the JVM and associated libraries along with any program to their production servers. This was an amount of operational complexity and resource overhead that was as undesirable a…

Also with GraalVM you may get away with building a native image which doesn't require a JVM at all.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#44

Post by tyingq »

austincheney wrote: post #39
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

> This entire article is a pretty damning report on JavaScript in general How so?

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task.

And that the operational improvement once written in Rust was notable enough to write a paper.

Imagine the K8S team porting from Go to some other language for similar reasons.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#45

Post by masklinn »

nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

I would expect that to be par for the course for most languages. The more dynamic the more problematic, but it stands to reason that the less you can check for and enforce statically the more will eventually blow up at runtime.

Resource usage is similar though not exactly aligned e.g. Haskell has significant ability to statically enforce invariants and handle error conditions, but the complex runtime and default laziness can make resource usage difficult to predict.

I'd guess OCaml would also have done well in the comparison as it too combines an extensive type to system which is difficult to bypass with an eager execution model.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#46

Post by platz »

twiss wrote: post #23
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

They also state that writing the service in Node took them an hour, two days for Go, and a week for Rust. Even taking into account their unfamiliarity with the language, it's probably fair to say that when switching to Rust, you'll usually spend more time writing and less time debugging. Whether that trade-off is worth it depends on the project.

> about a week to get up to speed in the language

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#48

Post by austincheney »

tyingq wrote: post #44
austincheney wrote: post #39

Earlier quoted context omitted.

> This entire article is a pretty damning report on JavaScript in general How so?

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

I guess I just don't see it. NPM is a massive infrastructure where one little piece was rewritten in Rust and then the Rust team wrote a promotional paper about it. The paper isn't bad, but it also isn't a technical white paper.

I don't see how any of this is critical of JavaScript, which isn't even really discussed in the paper and still runs the rest of the infrastructure. If anything the paper is more damning of C, C++, and Java but I still think damning is far too extreme to describe what the paper said.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#49

Post by platz »

masklinn wrote: post #45
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

I would expect that to be par for the course for most languages. The more dynamic the more problematic, but it stands to reason that the less you can check for and enforce statically the more will eventually blow up at runtime. Resource usage is similar though not exactly aligned e.g. Haskell has significant ability to statically enforce invariants and handle error conditions, but the complex runtime and default lazi…

Default laziness in Haskell is not as big a problem as is made out to be. For someone like NPM though, Haskell's current GC would probably be too much of a bottleneck. Haskell's GC is tuned for lots of small garbage and does not like a large persistent working-set. But this has nothing to do with laziness.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#50

Post by atoav »

nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject

Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version).

When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file. This means if it compiles for you, it should compile on every other machine too.

A cargo.toml allows you also to use (public or private) repositories as a sorce for a library, specify wildcard version numbers to only select e.g. versions newer than 1.0.3 and older than 1.0.7 etc.

Because the compiler will show you unused dependencies you never really end up including anything you don't use. In practice this system does not only work incredibly well, but is also very comfortable to use and isolates it self from the system it is running on quite well.

I really wish Python also had something like this. Pipenv is sort of going into that direction, but it is nowhere near cargo in functional terms.

Post reply on HN