We 30x'd our Node parallelism • Hacker News Board

Re: We 30x'd our Node parallelism

#51

Post by bjacokes » Fri, Dec 13, 2019, 4:08 PM UTC

rauchp wrote: post #8

That was an interesting read, thanks for linking to it. It's hard finding articles online discussing Node and performance, most people just dismiss it as an unviable option due to scale and speed concerns. 30x really is quite the jump though. > Each Node worker runs a gRPC server Not going to lie, this kind of surprised me. When I think of a Node backend I think of ExpressJS. Not because I think Express is better, bu…

Our integrations are primarily written in Node, which was the original language used for everything at Plaid. Almost all of those original services (except for integrations) have been migrated to Go or Python at this point. We've standardized on gRPC as our wire format, so we stayed consistent and used gRPC in Node.

With perfect hindsight, it's a fair point that all the pros and cons could net out to another language being best for our integrations. Integrations are the largest and most quickly-changing codebase at Plaid, so such a migration would be a massive undertaking. We definitely didn't want to block scalability improvements on doing a language migration.

Re: We 30x'd our Node parallelism

#52

Post by sergiotapia » Fri, Dec 13, 2019, 4:09 PM UTC

7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

At some point you have to take a step back and realize you've grown beyond your tech and reach out for something else. Elixir sounds like a great fit for these problems.

For example Discord reached out to Rust and built tiny Rust components that are called from Elixir for their server user list. Some servers have 200,000+ people online, and Elixir wasn't cutting it performance wise. Rust, boom now it works.

Re: We 30x'd our Node parallelism

#53

Post by coddle-hark » Fri, Dec 13, 2019, 4:09 PM UTC

dmitriid wrote: post #37

davedx wrote: post #27

Earlier quoted context omitted.

> trivially blocked by very simple programmer errors Can you give an example please? I think it's much easier to block a thread with C#'s async programming model than node's...

Node only has one thread. Everything else follows.

No it doesn’t. We’ve had good models for concurrency in single-threaded systems for a while now.

Re: We 30x'd our Node parallelism

#54

Post by spamizbad » Fri, Dec 13, 2019, 4:10 PM UTC

coddle-hark wrote: post #41

duxup wrote: post #19

Earlier quoted context omitted.

That was my thought to. They've got a problem where they've got no idea what a given transaction costs and some unpredictable amount of transactions result in some serious work that holds up the event queue. God knows they could be waiting for some reel to reel tape to spin up somewhere...

The whole point of async I/O is to be able to do something useful while waiting for tape to spin up. I don’t buy it.

But you need to know if you can do that something first, or if you've done that something too many times in the last N minutes (and could get blocked, forcing thousands of other somethings to get endlessly queued). Or if that something could take too long, and actually you could be doing 200 other somethings in the same time etc. It's not that simple.

Re: We 30x'd our Node parallelism

#55

Post by phoe-krk » Fri, Dec 13, 2019, 4:12 PM UTC

eshyong wrote: post #46

phoe-krk wrote: post #24

Earlier quoted context omitted.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

I mean, running multiple node runtimes (aka multiprocessing) actually sounds like a reasonable compromise for parallelism. That's the standard solution for dynamic languages without great multithreading support. If you needed great multithreading support then Node probably wasn't the right choice for you in the first place, but for most applications, it's probably fine. However, running multiple containers for parall…

Yet this is the case mentioned in the article.

> We were running 4,000 Node containers (or "workers") for our bank integration service.

Re: We 30x'd our Node parallelism

#56

Post by jerf » Fri, Dec 13, 2019, 4:12 PM UTC

phoe-krk wrote: post #42

earthboundkid wrote: post #31

Earlier quoted context omitted.

Async is just modern cooperative multitasking, and just like the 90s, it's easy to accidentally lock the whole system.

We are no longer in the 90s. The code has increased in volume a hundredfold and it comes from literally everywhere. You can no longer trust everything on your machine or your network to be bug-free or otherwise non-hostile. Creating a system in the 21st century that tries to follow ideals from the 90s gives us the kind of idiotism that we can witness here.

I think you may have misinterpreted earthboundkid; the claim isn't that it worked in the 90s, the claim is that it was already broken in the 90s.

You are otherwise on the right track, though Node does technically have one advantage, which is that it is a cooperatively-scheduled island in a preemptively-scheluded overall OS. In the 1990s, when the cooperatively-scheduled program was not cooperative, you locked the machine, not the process [1]. There is a reason why Apple went very aggressive with the OSX rewrite; the previous Systems had basically written themselves into a corner where they had to use cooperative multitasking because so much code made use of the implicit promises it provides, yet they could no longer afford to compete with Microsoft if they didn't get off it it, because the complexity just kept going up, up, up and the problem was going to continue getting exponentially worse.

For a Node program, you only have to account for the Node program itself, not everything running on the computer. Still, you're in the same exponentially-growing-complexity trap (with a very initially-safe-seeming low exponent, but it still gets you in the end), you just reset yourself back to a point earlier on the curve.

[1] There are various details, caveats, interrupts, etc, the picture is more complicated than one sentence can convey, but the principle still held and it was still possible to wedge the machine fairly badly for varying periods of time with simple bad code.

Re: We 30x'd our Node parallelism

#57

Post by james-mcelwain » Fri, Dec 13, 2019, 4:13 PM UTC

phoe-krk wrote: post #24

nicoburns wrote: post #9

Earlier quoted context omitted.

> We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. So they fixed the issue that some requests blocked... by making all requests blocking.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

Are you seriously arguing against event loops as a category? Blocking in non-blocking code is going to be an issue whether the language is C or JavaScript (duh).

Re: We 30x'd our Node parallelism

#58

Post by abalone » Fri, Dec 13, 2019, 4:13 PM UTC

bjacokes wrote: post #30

7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

There are a couple of reasons that the legacy scaling model was viable for us. As mentioned in the post, only 1/10 of our traffic was from the API, which gave us a roundabout way to scale by diverting resources. And it's only viable to use this model of scaling when the business value of a request is high – we were originally quite happy to spin up more containers when we reached our scaling limit. That's the pragmat…

> In terms of what issues caused us to move away from parallelism in the first place, it was all the CPU-bound stuff that you might expect: ReDoS-style issues, post-processing arrays in very large edge cases, programmer error, etc.

But these are not parallelism problems. These are single threading problems, which the core problem with Node.js, not parallelism in general. Hence I think the question stands: why did you choose node for this?

Re: We 30x'd our Node parallelism

#59

Post by phoe-krk » Fri, Dec 13, 2019, 4:13 PM UTC

JoeAltmaier wrote: post #44

phoe-krk wrote: post #24

Earlier quoted context omitted.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

Multithreading is still possible - for clarity of code. Multiple processing threads are running in parallel, but only one at a time. This is a subset of useful applications, but not totally worthless.

Yes, it is possible, and actually works - unless one of your threads blocks the event loop and prevents all others from running. This way it takes a single locking thread to halt the whole system, whereas with standard threads a single stalling thread does not cause all others to starve. This is a risk in Node that multiple other runtimes of other languages do not have.

Re: We 30x'd our Node parallelism

#60

Post by phoe-krk » Fri, Dec 13, 2019, 4:15 PM UTC

james-mcelwain wrote: post #57

phoe-krk wrote: post #24

Earlier quoted context omitted.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

Are you seriously arguing against event loops as a category? Blocking in non-blocking code is going to be an issue whether the language is C or JavaScript (duh).

For these kinds of programming, yes, I argue against it. A single stalling function in Node deadlocks the whole system; a single stalling thread in the C++ model still permits other threads to run. This is a risk that is completely avoidable by not using languages which require event loops at their core.