Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We 30x'd our Node parallelism

blog.plaid.com

Post reply on HN
Print view View original

41–50 of 261 posts

Re: We 30x'd our Node parallelism

#41

Post by coddle-hark »

duxup wrote: post #19
spamizbad wrote: post #14

The only way this makes sense to me is if they have to contend with lots of expensive parsing, event sequencing, and throttling requirements. Payment APIs, bank websites, etc can be quite byzantine. I could understand how one might code yourself into a corner with a monolithic node app and basically just say "F-it, we're doing this synchronously!" I don't even think it's a terribly bad thing to do assuming it favors…

That was my thought to. They've got a problem where they've got no idea what a given transaction costs and some unpredictable amount of transactions result in some serious work that holds up the event queue. God knows they could be waiting for some reel to reel tape to spin up somewhere...

The whole point of async I/O is to be able to do something useful while waiting for tape to spin up.

I don’t buy it.

Re: We 30x'd our Node parallelism

#42

Post by phoe-krk »

earthboundkid wrote: post #31
phoe-krk wrote: post #5

Earlier quoted context omitted.

> I thought one of the key selling points with Node was an fully async standard library, enabling better scaling in process. We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. The fact that Node ships a fully asynchronous standard library doesn't in any way fix the fact that Node is a runtime for a language that itself is a mi…

Async is just modern cooperative multitasking, and just like the 90s, it's easy to accidentally lock the whole system.

We are no longer in the 90s. The code has increased in volume a hundredfold and it comes from literally everywhere. You can no longer trust everything on your machine or your network to be bug-free or otherwise non-hostile.

Creating a system in the 21st century that tries to follow ideals from the 90s gives us the kind of idiotism that we can witness here.

Re: We 30x'd our Node parallelism

#43

Post by wiredfool »

earthboundkid wrote: post #31
phoe-krk wrote: post #5

Earlier quoted context omitted.

> I thought one of the key selling points with Node was an fully async standard library, enabling better scaling in process. We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. The fact that Node ships a fully asynchronous standard library doesn't in any way fix the fact that Node is a runtime for a language that itself is a mi…

Async is just modern cooperative multitasking, and just like the 90s, it's easy to accidentally lock the whole system.

Yeah, I remember just how nice it was going from Cooperative MT to Preemptive multi tasking -- the general view was that anything that only did Cooperative MT was just a Toy.

I'd bet that the orders of magnitude of speed from Moore's law did in CMT by making PMT doable without a huge speed hit.

Nothing I've seen from async is cleaner, easier to maintain, or better from a cognitive load POV. It's just more efficient for certain types of loads because you're being a consenting adult and not breaking things.

Re: We 30x'd our Node parallelism

#44

Post by JoeAltmaier »

phoe-krk wrote: post #24
nicoburns wrote: post #9

Earlier quoted context omitted.

> We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. So they fixed the issue that some requests blocked... by making all requests blocking.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

Multithreading is still possible - for clarity of code. Multiple processing threads are running in parallel, but only one at a time. This is a subset of useful applications, but not totally worthless.

Re: We 30x'd our Node parallelism

#45

Post by duxup »

coddle-hark wrote: post #41
duxup wrote: post #19

Earlier quoted context omitted.

That was my thought to. They've got a problem where they've got no idea what a given transaction costs and some unpredictable amount of transactions result in some serious work that holds up the event queue. God knows they could be waiting for some reel to reel tape to spin up somewhere...

The whole point of async I/O is to be able to do something useful while waiting for tape to spin up. I don’t buy it.

The article certainly raises more questions than answers that's for sure.

Re: We 30x'd our Node parallelism

#46

Post by eshyong »

phoe-krk wrote: post #24
nicoburns wrote: post #9

Earlier quoted context omitted.

> We still have an event loop that is trivially blocked by very simple programmer errors, destroying the whole advantage that you describe here. So they fixed the issue that some requests blocked... by making all requests blocking.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

I mean, running multiple node runtimes (aka multiprocessing) actually sounds like a reasonable compromise for parallelism. That's the standard solution for dynamic languages without great multithreading support. If you needed great multithreading support then Node probably wasn't the right choice for you in the first place, but for most applications, it's probably fine.

However, running multiple containers for parallelism sounds a little bit crazy. In the worst case, each container may be running on its own server, but even assuming multiple containers per host, I'm guessing they were running an insignificant number of instances, which is probably why they were able to save $300k in server costs.

Re: We 30x'd our Node parallelism

#47

Post by markstos »

spamizbad wrote: post #14

The only way this makes sense to me is if they have to contend with lots of expensive parsing, event sequencing, and throttling requirements. Payment APIs, bank websites, etc can be quite byzantine. I could understand how one might code yourself into a corner with a monolithic node app and basically just say "F-it, we're doing this synchronously!" I don't even think it's a terribly bad thing to do assuming it favors…

Their velocity might have been slowed by figuring out how to manage 4,000 containers effectively. If they had dealt with managing concurrency effectively sooner, they would need 30x less containers-- 133.

Re: We 30x'd our Node parallelism

#48

Post by nosianu »

They write (somewhere in the middle)

> Since V8 implements a stop-the-world GC, new tasks will inevitably receive less CPU time, reducing the worker’s throughput

But there is this Google blog post vom January 2019:

https://v8.dev/blog/trash-talk

> Over the past years the V8 garbage collector (GC) has changed a lot. The Orinoco project has taken a sequential, stop-the-world garbage collector and transformed it into a mostly parallel and concurrent collector with incremental fallback.

So I guess they used an older node.js version. The current LTS version is 12.x and it is from around the middle of this year.

---

PS: If the blog author reads this, there is an accessibility problem with the Google-hosted inline images. If I try - without ad blocker - in an anonymous window I see none of the inline images. Logged into Google with my own account I can see some but not all the images. Apparently which images I can see depends on being logged in to my Google account? I also tried IE Edge just to see if the browser makes a difference - no inline images visible there either.

Re: We 30x'd our Node parallelism

#49

Post by meritt »

Scarbutt wrote: post #33

I don't want to be that guy, but why did they start with nodejs for something like this instead of using the JVM or Go?

My guess is because their system is primarily issuing HTTP requests and extracting data out of responses: html, xml, json, plaintext, etc. Web scraping is a messy business and using a language that allows you to be flexible with string manipulation and types goes a long way toward sanity.

Re: We 30x'd our Node parallelism

#50

Post by mnutt »

I’d be curious to hear more about the circumstances that ended up with a blocked runloop. Are there hundreds of junior engineers, or perhaps third parties writing code that you don’t control? I have seen people accidentally write blocking code, but not at such an egregious rate that we couldn’t catch it in code review, or at worst the runloop detector would alert on it in prod and we would roll back the deploy.

For instances where you actually know you need lots of CPU, there are now strategies for offloading that specific work, although they have taken a while to get nice and easy to use.

Post reply on HN