We 30x'd our Node parallelism
blog.plaid.com
We 30x'd our Node parallelism
1–10 of 261 posts
Re: We 30x'd our Node parallelism
#2> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,000 concurrent requests, the system did not gracefully scale.
I can't be the only person who reads stories like this and wonders how they arrived at that solution in the first place?
Failing to scale because their previous approach to scaling was a worker per request, a model which was roundly moved away from, because that's how CGI and Apache modules worked and it didn't scale well.
I thought one of the key selling points with Node was an fully async standard library, enabling better scaling in process.
But then you read stories like this, and I find it hard to relate to the original problem.
Re: We 30x'd our Node parallelism
#3I’d be curious to hear your reevaluation of moving this to Lambda after some of the major announcements during re:invent. My guess is some of the reasons you went ECS have been addressed with these announcements. Obviously some of the new features are still preview, but would be interested to hear your analysis none the less.
Re: We 30x'd our Node parallelism
#4> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…
Agreed. This is the second scratch my head moment from the Plaid engineering team blog recently.
Re: We 30x'd our Node parallelism
#5> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…
> I thought one of the key selling points with Node was an fully async standard library, enabling better scaling in process.
We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here.
The fact that Node ships a fully asynchronous standard library doesn't in any way fix the fact that Node is a runtime for a language that itself is a mistake.
Re: We 30x'd our Node parallelism
#6Does node have something similar to how apcu is used with PHP?
That is, an mmap based kv store so that if you choose to run more than one node process on a single server, it has a fast kv cache?
I'm aware you can use redis or similar, but a simple mmap kv store is simpler and faster for a single server use case.
Re: We 30x'd our Node parallelism
#7> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…
I got the same feeling. We use node and usually split to 500 concurrent requests per process.
Still interesting...
Re: We 30x'd our Node parallelism
#8That was an interesting read, thanks for linking to it. It's hard finding articles online discussing Node and performance, most people just dismiss it as an unviable option due to scale and speed concerns. 30x really is quite the jump though.
> Each Node worker runs a gRPC server
Not going to lie, this kind of surprised me. When I think of a Node backend I think of ExpressJS. Not because I think Express is better, but because it's been pushed around in the past few years as the fastest, simplest way of running a backend.
Yet, if you're going to be running a gRPC server, why not use a more performant language with better multithreading support? I thought this article was about them optimizing a grandfathered-in solution (such as Express), but I can't tell why they built out a gRPC server in Node in the first place.
Re: We 30x'd our Node parallelism
#9> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…
> I thought one of the key selling points with Node was an fully async standard library, enabling better scaling in process. We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. The fact that Node ships a fully asynchronous standard library doesn't in any way fix the fact that Node is a runtime for a language that itself is a mi…
> We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here.
So they fixed the issue that some requests blocked... by making all requests blocking.
Re: We 30x'd our Node parallelism
#10"Only 10% of Plaid's data pulls involve a user who is present"
Since they provide an API, it seems like some of the calls where they think a user isn't present might actually have one present.