Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We 30x'd our Node parallelism

blog.plaid.com

Post reply on HN
Print view View original

61–70 of 261 posts

Re: We 30x'd our Node parallelism

#61

Post by winrid »

phoe-krk wrote: post #24
nicoburns wrote: post #9

Earlier quoted context omitted.

> We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. So they fixed the issue that some requests blocked... by making all requests blocking.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

Running a Node worker for each thread is standard practice.

No different than having a dedicated threadpool for asynchronous programming on the JVM.

Yes blocking the event loop is easy. No it's not THAT easy. I've never done it because you think about it while writing code. It's part of the environment. I have had to fix lots of reports etc that try to load up the world and iterate through it in a loop that doesn't yield to the event loop. It's possible to never make that mistake by understanding your environment (just like how managing pointers in C is "hard").

Re: We 30x'd our Node parallelism

#62

Post by Slartie »

coddle-hark wrote: post #41
duxup wrote: post #19

Earlier quoted context omitted.

That was my thought to. They've got a problem where they've got no idea what a given transaction costs and some unpredictable amount of transactions result in some serious work that holds up the event queue. God knows they could be waiting for some reel to reel tape to spin up somewhere...

The whole point of async I/O is to be able to do something useful while waiting for tape to spin up. I don’t buy it.

But the whole point of synchronous I/O is to isolate the programmer from having to think about that spinning tape up takes a non-zero time. I have a feeling that this gets lost sometimes in all that "async I/O is the GREATEST!" craze.

Async is nice - if you can handle it. But this is not easy to do in complex systems and processes. It is certainly easier to work with an old-fashioned process that blocks when waiting for whatever you need to wait for, and just scale by letting the OS run lots of those in parallel. Sure, it's less efficient. But it's easier for the devs to handle.

I just read the hidden undertone of this article as "our devs aren't that smart after all".

Re: We 30x'd our Node parallelism

#63

Post by coddle-hark »

deedubaya wrote: post #36

A good example of avoiding premature optimization. I'd imagine delaying tackling this problem freed them up to tackle problems that impact users.

This only holds if they didn’t pour hours into the original solution. Setting up and managing 4000 node services doesn’t sound like a quick hack.

Re: We 30x'd our Node parallelism

#64

Post by asdfman123 »

7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

> I can't be the only person who reads stories like this and wonders how they arrived at that solution in the first place?

Here's how it probably worked: they liked Node, they liked containers, they put Node into containers and it worked, and they stuck with it as the user base grew.

Re: We 30x'd our Node parallelism

#65

Post by neillyons »

wiredfool wrote: post #43
earthboundkid wrote: post #31

Earlier quoted context omitted.

Async is just modern cooperative multitasking, and just like the 90s, it's easy to accidentally lock the whole system.

Yeah, I remember just how nice it was going from Cooperative MT to Preemptive multi tasking -- the general view was that anything that only did Cooperative MT was just a Toy. I'd bet that the orders of magnitude of speed from Moore's law did in CMT by making PMT doable without a huge speed hit. Nothing I've seen from async is cleaner, easier to maintain, or better from a cognitive load POV. It's just more efficient f…

What language were you using that had preemptive multi tasking?

Re: We 30x'd our Node parallelism

#66

Post by Turbots »

7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

Using an interpreted language and complaining about performance is always a funny thing to read.

Use Java, C++, Rust, Go or C if you want pure performance.

Use Java with something like Spring + a Reactive stack to scale up to much higher concurrent numbers of requests. You'll have the same easy programming model that Node provides and you'll have both performance from the language and the more efficient execution.

If you're worried about the startup times and memory footprint that comes with a JVM based application (although this has improved substantially the last few years), go to a natively compiled language like C, C++, Rust or Go. Or compile your Java apps using GraalVM native images.

Re: We 30x'd our Node parallelism

#67

Post by kevstev »

I was building scalable node applications a few years ago for a very large e-commerce player- millions of customers. I think node.js is a great platform, but its apparent simplicity means there are hordes, and I mean like 90+% of the community, that can "just get things done" without understanding what is going on under the hood at all. And to be fair, for most startupy types of companies that need to iterate fast, that is what you want to optimize for.

My interview screening question was pretty simple- "Is node.js single threaded or multithreaded?" And to most, they spit back the blogspam headline- "Single threaded!" I think the most correct answer is "its complicated" but would accept that because most people would say that is the "right" answer. So I would follow up with- "what exactly happens in a default installation if we have say... 5 requests come in at exactly the same time to just return some static content from disk?" (Node's default threadpool is 4). And here is where you could see their understanding just fell apart. Some would say they would be handled entirely synchronously, others completely in parallel- but then had no idea what the cause of the parallelism was. Very few actually understood that node is an event loop executing javascript backed by a threadpool for async operations.

Before reading this post, I was like eh this is a waste of time- its typical medium bullshit- they almost certainly found they were doing some blocking call in the event loop and then removed it and voila, 30x speedup. It was interesting because it was a lot worse! They spent all this time and hard work figuring out everything but what was taking so long in the event loop, and it seems that was the last place they actually looked.

Anyway, node can be a highly scalable platform (https://changelog.com/podcast/116) but you need to understand it or else it will bite you in the foot. When I was last doing this stuff, upwards of 80% of our time was being spent essentially just JSON.parse()'ing, and we were looking to move to protobufs to avoid that.

Re: We 30x'd our Node parallelism

#68

Post by asdfman123 »

api wrote: post #23
7777fps wrote: post #2

> We were running 4,000 Node containers (or "workers") for our bank integration service. The service was originally designed such that each worker would process only a single request at a time. This design lessened the impact of integrations that accidentally blocked the event loop, and allowed us to ignore the variability in resource usage across different integrations. But since our total capacity was capped at 4,0…

I wonder what percentage of the massive compute power of huge cloud data centers is spent just chugging away on ugly clunky hacks to run bad code?

Quite a large percentage. I worked on a site that got 1 request per second and they were able to handle it by spinning up like 20 VMs. Turns out they were just using Entity Framework wrong. Whoops.

But also, though, you have to consider that most places aren't Plaid, and most places developer time is more expensive than throwing an extra machine at the problem.

Re: We 30x'd our Node parallelism

#70

Post by [deleted] »

eshyong wrote: post #46
phoe-krk wrote: post #24

Earlier quoted context omitted.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

I mean, running multiple node runtimes (aka multiprocessing) actually sounds like a reasonable compromise for parallelism. That's the standard solution for dynamic languages without great multithreading support. If you needed great multithreading support then Node probably wasn't the right choice for you in the first place, but for most applications, it's probably fine. However, running multiple containers for parall…

[deleted]
Post reply on HN