Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We 30x'd our Node parallelism

blog.plaid.com

Post reply on HN
Print view View original

101–110 of 261 posts

Re: We 30x'd our Node parallelism

#101

Post by tyingq »

bjacokes wrote: post #34
tyingq wrote: post #10

"Only 10% of Plaid's data pulls involve a user who is present" Since they provide an API, it seems like some of the calls where they think a user isn't present might actually have one present.

We thread knowledge of whether a data pull was initiated by the API or by our cron-style service into our load-balancing layer, so this ends up being pretty straight-forward.

Ahh, got it. The "present and linking their account" part threw me off. Sounded like only the "linking" call was getting the fast lane.

Re: We 30x'd our Node parallelism

#102

Post by jrockway »

rauchp wrote: post #8

That was an interesting read, thanks for linking to it. It's hard finding articles online discussing Node and performance, most people just dismiss it as an unviable option due to scale and speed concerns. 30x really is quite the jump though. > Each Node worker runs a gRPC server Not going to lie, this kind of surprised me. When I think of a Node backend I think of ExpressJS. Not because I think Express is better, bu…

The Node gRPC implementation is fine. It uses the C++ implementation which is the gold standard. It has Prometheus and OpenTracing interceptors. You basically give nothing up by using it, if your team wants to write a language that runs on node.

Re: We 30x'd our Node parallelism

#103

Post by NathanKP »

kevstev wrote: post #67

I was building scalable node applications a few years ago for a very large e-commerce player- millions of customers. I think node.js is a great platform, but its apparent simplicity means there are hordes, and I mean like 90+% of the community, that can "just get things done" without understanding what is going on under the hood at all. And to be fair, for most startupy types of companies that need to iterate fast, t…

This is why I recommend that anyone running Node in production use a tracing tool like New Relic. It's super easy to see what is blocking the event loop. Just choose a duration (say 10ms) and look for any execution spans that are longer than that duration.

Ideally you want to be yielding back to the event loop at least every 1 ms. Anything that takes too long without yielding will show up as a latency delay before your code is able to start handling a new request (technically a background thread in Node.js will pick up the request, but your code won't start executing in response to it until you yield back to the event loop again).

To be honest the more difficult thing to diagnose sometimes is event loop overburdening. If each of your execution spans are taking 1ms, then you can only do a max of 1000 of them per second (assuming there was no delay between executions, but there is). So if you are trying to handle a large number of requests per second the event loop may end up with say 1005 execution spans per second that it needs to execute to handle that request volume. Because you can't do 1005ms of work in 1000ms the extra work will queue up.

So gradually you will end up with 5 backlogged execution spans stacking up per second. Each second you will get 5ms more latency. The overall request latency will just gradually increase and increase as work gets further and further delayed in the queue.

Overall I just think of Node.js as a fancy CPU scheduler. As long as you give it even, decently sized chunks of work to schedule, and you don't give it too many to schedule you will be fine. Anyway I'm a huge fan of Node.js but yeah its easy to fall into some gotcha's if you don't study how it works. The simplicity is a bit misleading

Re: We 30x'd our Node parallelism

#104

Post by kevstev »

x86_64Ubuntu wrote: post #96
kevstev wrote: post #67

I was building scalable node applications a few years ago for a very large e-commerce player- millions of customers. I think node.js is a great platform, but its apparent simplicity means there are hordes, and I mean like 90+% of the community, that can "just get things done" without understanding what is going on under the hood at all. And to be fair, for most startupy types of companies that need to iterate fast, t…

What does happen with the 5th request.

It will get queued, until one of the 4 requests in front of it has its task, to return the file, complete.

Re: We 30x'd our Node parallelism

#105

Post by namibj »

tyingq wrote: post #21
CamouflagedKiwi wrote: post #18

Earlier quoted context omitted.

The other 90% are not triggered by the API, they are "periodic transaction updates" - presumably they refresh once a day or something.

Yeah, I read that, but it's not clear exactly what those calls are. It sorta sounds like making assumptions on how their users are using the API. In fact, it sounds like they think "linking an account" is the only "user present" API call: "Only 10% of Plaid's data pulls involve a user who is present and linking their account to an app"

No, I'd read this as "linking their account to an app" meaning that the plain account's API credentials are configured in the app, so the app can call the plaid API (presumably interactively on user interaction).

Re: We 30x'd our Node parallelism

#106

Post by NathanKP »

x86_64Ubuntu wrote: post #96
kevstev wrote: post #67

I was building scalable node applications a few years ago for a very large e-commerce player- millions of customers. I think node.js is a great platform, but its apparent simplicity means there are hordes, and I mean like 90+% of the community, that can "just get things done" without understanding what is going on under the hood at all. And to be fair, for most startupy types of companies that need to iterate fast, t…

What does happen with the 5th request.

It waits in a queue in a background thread in the node.js http library until your code can execute to handle it. So if your code takes a long time before returning back to the event loop the request will just wait in that queue for a long time before the next opportunity for the event loop to execute code in response to the event.

Re: We 30x'd our Node parallelism

#107

Post by godot »

tyingq wrote: post #6

Does node have something similar to how apcu is used with PHP? That is, an mmap based kv store so that if you choose to run more than one node process on a single server, it has a fast kv cache? I'm aware you can use redis or similar, but a simple mmap kv store is simpler and faster for a single server use case.

I totally see what you mean, coming from a PHP world myself a few years ago. The key thing to note is that node.js (like many other languages including Java) starts a server process that basically does not stop until you explicitly restart it (or it crashes); unlike PHP where every request starts a brand new process on a clean slate (hence needing APCu to store a local memory cache per server). Meaning, what you can accomplish with APCu in PHP can be trivially accomplished by a simple Object in node.js (i.e. a map/hash), by virtue of having a require cache (hence every time you require'd the lib it returns the same instance of the object).

If you want a simple open source lib to do exactly that for you and provide an easy to use API, you can use something like https://www.npmjs.com/package/tmp-cache .

Re: We 30x'd our Node parallelism

#108

Post by markandrewj »

crazygringo wrote: post #99
7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

Yeah, I don't get it either, at all. The original poster wrote below: > In terms of what issues caused us to move away from parallelism in the first place, it was all the CPU-bound stuff that you might expect: ReDoS-style issues, post-processing arrays in very large edge cases, programmer error, etc. But it's trivial (a single line) in Node to place breaks in CPU processing to allow the event loop to fire, and as for…

The issue is that many developers that are coming from synchronous programming don't get asynchronous programming. They could both improve the code by not writing blocking code, and also using something like the cluster module (https://nodejs.org/api/cluster.html).

Re: We 30x'd our Node parallelism

#109

Post by inglor »

kevstev wrote: post #67

I was building scalable node applications a few years ago for a very large e-commerce player- millions of customers. I think node.js is a great platform, but its apparent simplicity means there are hordes, and I mean like 90+% of the community, that can "just get things done" without understanding what is going on under the hood at all. And to be fair, for most startupy types of companies that need to iterate fast, t…

> Very few actually understood that node is an event loop executing javascript backed by a threadpool for async operations.

This is true, and that JavaScript is mostly a synchronous programming language with host environments that can provide asynchronisity.

A caveat though is that the most important part of I/O is network I/O (tcp/udp sockets) and Node uses real async operations there rather than a threadpool.

FS is just really hard to get right in a cross platform way and that's why it's on the threadpool. Some other stuff like dns is also famously on the threadpool but tcp sockets are not - it's a big part of why Node is fast.

Re: We 30x'd our Node parallelism

#110

Post by bjacokes »

abalone wrote: post #58
bjacokes wrote: post #30

Earlier quoted context omitted.

There are a couple of reasons that the legacy scaling model was viable for us. As mentioned in the post, only 1/10 of our traffic was from the API, which gave us a roundabout way to scale by diverting resources. And it's only viable to use this model of scaling when the business value of a request is high – we were originally quite happy to spin up more containers when we reached our scaling limit. That's the pragmat…

> In terms of what issues caused us to move away from parallelism in the first place, it was all the CPU-bound stuff that you might expect: ReDoS-style issues, post-processing arrays in very large edge cases, programmer error, etc. But these are not parallelism problems. These are single threading problems, which the core problem with Node.js, not parallelism in general. Hence I think the question stands: why did you…

It was chosen about 6 years ago when the product was first being developed, so most of us on the engineering team weren't around when the decision was made. The main choice we're making at this point is: what's the impact and ROI of a language migration vs getting Node to work as well as we can?
Post reply on HN