Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

rust-lang.org

Post reply on HN
Print view View original

131–140 of 307 posts

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#131

Post by tyingq »

derefr wrote: post #124
icebraining wrote: post #108

Earlier quoted context omitted.

We're not talking about average JS developers, we're talking about the NPM maintainers. EDIT: I did miss that. Sorry for the noise.

You missed the joke. Let me quote the GP again: > I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one. For this to be "ironic", having lots and lots of dependencies would actually have to be a Node.js "best practice." But it's not. It seems like a best practice to outsiders, for the same reason that setting `register_globals` seemed…

I'm not convinced. I see stuff like this fairly often: https://news.ycombinator.com/item?id=19290801 (Basically how it might be common to have 1000+ dependent files in a JS app)

From people that seem plenty intelligent and experienced.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#132

Post by Filligree »

tyingq wrote: post #99
Quenty wrote: post #92

Earlier quoted context omitted.

The paper makes it clear they were evaluating languages based upon efficiency. The rust implementation was more efficient than the JS one. A CPU bound service of course is bottlenecked at the CPU, and this benefits from efficiency. At scale, it makes sense to replace this with Rust. Javascript did the job, but did not provide the same efficiency as Rust.

I'm not clear on why this is particularly CPU heavy, though: "the authorization service that determines whether a user is allowed to, say, publish a particular package"

Lots of users. Anything will be CPU-heavy if you give it enough work.

Except the things that end up being memory-bound instead, but the NPM database isn't large enough for that.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#133

Post by tyingq »

pjmlp wrote: post #97
tyingq wrote: post #84

Earlier quoted context omitted.

Commercial development or test use isn't free. It's only free for personal, noncommercial use. So I'd argue it isn't so easy to understand.

How it isn't? You just explained it quite well. Not to mention that the Web is now full of discussions around this, with official posts from Oracle, Red-Hat, Amazon, IBM, Azul, Microsoft explaining how to go forward.

Figure out what you would pay if you used it in an elastic ECS setup. Using only Oracle documentation as a guide :)

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#134

Post by colejohnson66 »

steveklabnik wrote: post #70
nikeee wrote: post #64

Anyone knows if they considered .NET Core? It has dependency management, can be deployed as a self-contained binary and is memory safe. Seems to match the requirements for me.

https://news.ycombinator.com/item?id=19295166

> This stuff happened before, or at least around the time, that .NET Core was released. So it either didn’t exist or was an extremely new option, at least.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#135

Post by geofft »

amluto wrote: post #94
faitswulff wrote: post #20

Earlier quoted context omitted.

The whitepaper notes that almost 9 billion NPM packages are downloaded per week, so I don't see anything laughable about needing good monitoring.

Which is roughly the equivalent of every single human being downloading two npm packages per week. To me, this suggests that the real problem is that too many packages are being downloaded.

I think this is a natural result of two things which should be appealing to fans of old-school UNIX philosophy:

- NPM is intentionally suited to lots of small libraries that do one thing and do it (hopefully) well, and composing those libraries in useful ways. Whereas systems like Debian have scaling limits with large numbers of packages, NPM tries hard to avoid this so that one hundred ten-line packages are as reasonable as a single thousand-line package.

- CI systems aim for reproducibility by deploying from source and having declarative configurations, in much the way that most distro package builds happen in a clean-room environment.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#136

Post by bluejekyll »

lllr_finger wrote: post #118
api wrote: post #115

Earlier quoted context omitted.

The same is generally true for things written in Go, Java, C#, etc. Strong typing and memory safety eliminate huge classes of common bugs.

Do you mean static typing?

The distinction in the vernacular between statically typed and strongly typed languages is so narrow at this point that pointing it out is a bit pedantic.

I would say Rust is both a strongly typed language and a statically typed language. The static type checking happens at compile time, and in general the types in use are strict and strongly typed at runtime.

But, even Rust allows you to cast types from one to another and use dynamic types determined at runtime.

Yes, most people would say that static typing is the primary advantage you get from the compiler in Rust.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#137

Post by phillipcarter »

steveklabnik wrote: post #47

Hey folks! This is part of our whitepaper series. This means that the audience is CTOs of larger organizations, and so the tone and content are geared for that, more than HN. Please keep that in mind!

Good overall whitepaper and I like to see efforts like these.

One are I felt was missing is some data here:

"It keeps resource usage low without the possibility of compromising memory safety. "

How did the resource usage compare with the Go and node rewrites? What metrics were used under which workload? Benchmarks are never perfect but I think a CTO-level person would like to see a table of results like that.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#138

Post by earenndil »

atoav wrote: post #50
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

Dub definitely does this (pretty much exactly the same, dub.json = Cargo.toml, dub.selections.json = Cargo.lock), and afaik cpan does something similar.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#139

Post by bigcostooge2 »

kristopolous wrote: post #120
atoav wrote: post #50

Earlier quoted context omitted.

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

> Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). Why do people want this? The builds are no longer reproducible, security and edge case issues can come out of nowhere, api changes from an irresponsible maintainer can break things, network and resource failure can break the build, it's just a terrible idea. The proper use of a semvar system is entir…

Exactly this. You can tell actual developer experience from willingness to update a library. Any library.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#140

Post by ricardobeat »

tyingq wrote: post #131
derefr wrote: post #124

Earlier quoted context omitted.

You missed the joke. Let me quote the GP again: > I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one. For this to be "ironic", having lots and lots of dependencies would actually have to be a Node.js "best practice." But it's not. It seems like a best practice to outsiders, for the same reason that setting `register_globals` seemed…

I'm not convinced. I see stuff like this fairly often: https://news.ycombinator.com/item?id=19290801 (Basically how it might be common to have 1000+ dependent files in a JS app) From people that seem plenty intelligent and experienced.

That thread is about a module system. 1000 files != 1000 third-party dependencies.
Post reply on HN