Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

rust-lang.org

Post reply on HN
Print view View original

161–170 of 307 posts

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#161

Post by ambicapter »

derefr wrote: post #72
tyingq wrote: post #15

Earlier quoted context omitted.

I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one.

Not so ironic—just because a practice is common among the median engineer in an ecosystem, doesn’t mean it will be common among the most experienced engineers in that ecosystem. Node gets a bad rap mostly for the fact that it has tons and tons of inexperienced engineers using it (probably as one of their first programming languages.) Same reason PHP got a bad rap back when. You can build solid software in both, by fi…

Do you have any advice for PHP best practices? I'm starting a job that will probably have me using PHP...

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#162

Post by IsaacSchlueter »

gkop wrote: post #153
GordonS wrote: post #112

Earlier quoted context omitted.

That sounds pretty similar to NPM, as well as NuGet and Paket for .NET. TBH, it's the 'obvious' way for a package manager to work, so I'd be a little surprised if they didn't all work more or less the same?

You have to run npm ci instead of npm install to get npm to respect the lock file. I don’t consider that remotely obvious. And this feature was just added to npm last year, 8 years after npm was invented!

That is incorrect. Both `npm install` and `npm ci` respect the lock file, and if a lock file is present, will make the `node_modules` tree match the lock file exactly.

`npm ci` is optimized for a cold start, like on a CI server, where it's expected that `node_modules` will not be present. So, it doesn't bother looking in `node_modules` to see what's already installed. So, _in that cold start case_, it's faster, but if you have a mostly-full and up to date `node_modules` folder, then `npm install` may be faster, because it won't download things unnecessarily.

Another difference is that `npm ci` also won't work _without_ a `package-lock.json` file, which means it doesn't even bother to look at your `package.json` dependencies.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#163

Post by pjmlp »

patrec wrote: post #147
tyingq wrote: post #107

Earlier quoted context omitted.

Thanks! The talk: "The clusterfuck hidden in the Kubernetes code base" https://fosdem.org/2019/schedule/event/kubernetesclusterfuck...

One wonders if there'll be a similar talk in a year... > The audience walks away feeling empathetic that they aren’t alone in their journey to writing idiomatic Go and is now equipped with strong refactoring techniques developed by some of the world’s top engineers for the Kuberentes project. As an occasional user of kubernetes, minikube etc. it's not something I would have guessed to have been developed by the word'…

You would be surprised at the quality of Android tooling stable releases, to the point that now there is Project Marble in place to try to improve its image.

https://adtmag.com/articles/2019/01/17/project-marble.aspx

If you want to have some reading fun, check /r/androiddev/ every time there is a "stable" release.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#164

Post by jononor »

amluto wrote: post #94
faitswulff wrote: post #20

Earlier quoted context omitted.

The whitepaper notes that almost 9 billion NPM packages are downloaded per week, so I don't see anything laughable about needing good monitoring.

Which is roughly the equivalent of every single human being downloading two npm packages per week. To me, this suggests that the real problem is that too many packages are being downloaded.

Probably a lot of these downloads are from bots. Continuous Integration is very common in Node.js/JavaScript projects, so each git commit anyone with CI (and no dependency caching) will download lots of packages.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#165

Post by ISO-morphism »

atoav wrote: post #50
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

I think Poetry [1] is the most promising in the python build/dependency space. I've used pipenv and left dissatisfied.

[1] https://github.com/sdispater/poetry

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#166

Post by pjmlp »

tyingq wrote: post #148
pjmlp wrote: post #142

Earlier quoted context omitted.

Why should I? There is OpenJDK for that. But assuming that Oracle support is actually desired, here is what you are asking for. https://www.oracle.com/technetwork/java/java-se-support-road... https://www.oracle.com/java/java-se-subscription.html https://shop.oracle.com/apex/f?p=DSTORE:2:::NO:RIR,RP,2:PROD... Old Java dog here. But lets just forget about the efforts from AdoptaJDK, Red-Hat, Amazon, Azul and bash Oracl…

"here is what you are asking for" None of that helps you with what the charge is when you're elastic, or using multiples of hyperthreads that don't add up to a integer number of cores. I agree that OpenJDK is a better idea.

If you want to be fully sure and insist in using Oracle JDK, then do like any enterprise shop and call the sales team to get their written word, what is so hard about it?

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#167

Post by tyingq »

pjmlp wrote: post #163
patrec wrote: post #147

Earlier quoted context omitted.

One wonders if there'll be a similar talk in a year... > The audience walks away feeling empathetic that they aren’t alone in their journey to writing idiomatic Go and is now equipped with strong refactoring techniques developed by some of the world’s top engineers for the Kuberentes project. As an occasional user of kubernetes, minikube etc. it's not something I would have guessed to have been developed by the word'…

You would be surprised at the quality of Android tooling stable releases, to the point that now there is Project Marble in place to try to improve its image. https://adtmag.com/articles/2019/01/17/project-marble.aspx If you want to have some reading fun, check /r/androiddev/ every time there is a "stable" release.

I bet k8s would look different if it were written by developers used to a more limited set of cpu, io, and memory resources.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#168

Post by IsaacSchlueter »

tyingq wrote: post #44
austincheney wrote: post #39

Earlier quoted context omitted.

> This entire article is a pretty damning report on JavaScript in general How so?

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

I mean, we still write the overwhelming majority of our code in JavaScript. We port to Rust when CPU-heavy task becomes a bottleneck to the rest of the system. It's not as if this paper is saying (nor is it the case) that we've ported the whole registry to Rust. JS has lots of advantages we appreciate.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#169

Post by spdionis »

zukzuk wrote: post #59
atoav wrote: post #50

Earlier quoted context omitted.

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

npm (and yarn) literally does exactly all of this, via `npm init funkyproject` and `package-lock.json`.

Except that npm will gladly update your lock file when you run npm install which is insane.
Post reply on HN