Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

rust-lang.org

Post reply on HN
Print view View original

181–190 of 307 posts

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#181

Post by ddebernardy »

nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

Anecdotal, but I recently used Gulp in a project to run some css clean-up tasks as part of a build process.

The JS dependencies:

    "gulp"
    "gulp-clean-css"
    "gulp-postcss"
    "gulp-uglify"
    "autoprefixer"
    "postcss-uncss"
    "uncss"
The number of node modules: just over 400.

So I'm not at all surprised that this might create surprises when deploying JS services in production.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#183

Post by z3t4 »

fpgaminer wrote: post #160
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

I wrote and deployed a production service written in pre-1.0 Rust. In over three years of being deployed I never once had to touch that code. The infrastructure around it evolved several times, we even moved cloud providers in that time, but that particular service didn't need any changes or maintenance. It just kept chugging along. Perhaps Rust's name is apropos: your code will be so reliable that you won't need to…

Never touched code is seldom a sign of quality. But there's the old saying; if it ain't broken, don't fix it.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#184

Post by bloopernova »

paulddraper wrote: post #117
bloopernova wrote: post #14

Earlier quoted context omitted.

Well, speaking from experience with a JBoss application layer in a recent software project I worked on: New java versions do break existing libraries or apps, and need to be tested thoroughly. When the company hasn't budgeted for that expense, it becomes difficult to update. Often an architect or software team will insist on using the Oracle JVM rather than the included openjvm. That adds extra steps to download, sto…

> New java versions do break existing libraries or apps Have you used the Rust compiler? My experience tells me that any Github Rust project not updated in the last two years doesn't work with my Rust compiler. Whereas Java apps written a decade ago still compile and run on OpenJDK/Oracle often with zero or near zero changes. > Often an architect or software team will insist on using the Oracle JVM rather than the in…

It's probably just the lack of emote via text, but your reply comes off as confrontational. It's also seemingly replying to me about general Rust issues when I'm just sharing my experience of a recent Java project. I don't have much experience of Rust beyond reading some code every now and then when an interesting blog post pops up here.

I'm also not condoning the decisions made in the project I describe. In fact, those ancient decisions are responsible for a significant amount of stress and work for me right now.

I've absolutely had experience of good JVM based projects that had loose dependencies on libraries and jdk versions such that it ran just fine under the openjvm, and ran just fine when that package was updated.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#186

Post by c-smile »

adamnemecek wrote: post #12

I’ve been playing around with rust since it came out but only recently did I decide to use it for a part of a project. It’s a very pleasant language. I didn’t fight the borrow checker much (maybe due to prior experience). The language is nuts. It’s true what they say cargo is even better than the language, it’s just so easy to add packages to your project or to split your project into packages. Cargo is an amazing in…

> write non duplicated code ... Like how many string implementations are there across c code bases.

In one of projects I needed foo::string that can share data with foo::variant without copying the data each time. So foo::string was implemented as a COW string - smart pointer for foo::string_data. std::string simply does not work in such requirements.

So I am not sure I understand how cargo will help in this case. Either Cargo source repository will contain implementations of any possible permutation of requirements of strings or people will just use standard std::string.

The only feature that I need in C/C++ is unified ability to include libraries in code:

    #define PNG_APNG_REQUIRED 
    #include source "libs/png/png-amalgamated.c"
I am perfectly fine with downloading png.tar.gz manually and putting it in place where I need it.

In any case decision to include library to a product requires quite a lot of reasonings and architectural investigations.

For typical web front-end projects NPM or Cargo probably make sense. But for, say, NodeJs or Cargo itself they should not use any such automatic downloader.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#187

Post by adamnemecek »

lioeters wrote: post #180
adamnemecek wrote: post #12

I’ve been playing around with rust since it came out but only recently did I decide to use it for a part of a project. It’s a very pleasant language. I didn’t fight the borrow checker much (maybe due to prior experience). The language is nuts. It’s true what they say cargo is even better than the language, it’s just so easy to add packages to your project or to split your project into packages. Cargo is an amazing in…

> Modern development needs a workflow where you pull in a dependency, and work on it in tandem with your code. Achieving a good workflow for this is surprisingly hard. Well put! I've struggled with this exact situation, and although I figured out a setup that works for me, it's still not ideal. In my experience, npm the package manager doesn't enable such workflows reliably (yet).

I agree. A lot of times I end up just pushing dependencies to github and pulling from github in the main codebase.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#188

Post by db48x »

GordonS wrote: post #112
atoav wrote: post #50

Earlier quoted context omitted.

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

That sounds pretty similar to NPM, as well as NuGet and Paket for .NET. TBH, it's the 'obvious' way for a package manager to work, so I'd be a little surprised if they didn't all work more or less the same?

Everything has evolved to get to that point. I suppose if you start with something modern like npm then it's not obvious how bad the earlier ones are. Compare the good ones with composer, dpkg, rpm, apt or dnf, to name a few examples.

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#189

Post by StreamBright »

atoav wrote: post #50
nothrabannosir wrote: post #10

This entire article is a pretty damning report on JavaScript in general, but this sentence takes the cake (emphasis mine): > The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monito…

Rust's package managment (cargo) is the best thing I have ever seen of it's kind. The very basic thing you can do is: cargo new funkyproject Which creates a new barebones rust project called "funkyproject". Every dependency specified in it's Cargo.toml will be automatically downloaded at build (if there is a new version). When a build is sucessful the versions of said dependency will be saved into a Cargo.lock file.…

Well just like many other languages with sane environment (dependencies, building, etc.) management. I think this is the norm nowadays (D, Clojure, and so on).

Re: Case Study: Npm uses Rust for its CPU-bound bottlenecks [pdf]

#190

Post by andy_ppp »

Dylan16807 wrote: post #159
Filligree wrote: post #132

Earlier quoted context omitted.

Lots of users. Anything will be CPU-heavy if you give it enough work. Except the things that end up being memory-bound instead, but the NPM database isn't large enough for that.

> Anything will be CPU-heavy if you give it enough work. Not in a relative sense. If authorization is 5% of the work, scaling it leaves it at 5% of the work, and it's never a bottleneck. Authorization was being a significant bottleneck, not a tiny percent, and that is somewhat surprising.

Obviously authorisation will be a huge overhead compared to sendfile + nginx right? Am I misunderstanding what npm does?
Post reply on HN