Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We 30x'd our Node parallelism

blog.plaid.com

Post reply on HN
Print view View original

241–250 of 261 posts

Re: We 30x'd our Node parallelism

#241

Post by winrid »

Osiris wrote: post #81
winrid wrote: post #61

Earlier quoted context omitted.

Running a Node worker for each thread is standard practice. No different than having a dedicated threadpool for asynchronous programming on the JVM. Yes blocking the event loop is easy. No it's not THAT easy. I've never done it because you think about it while writing code. It's part of the environment. I have had to fix lots of reports etc that try to load up the world and iterate through it in a loop that doesn't y…

> Yes blocking the event loop is easy. No it's not THAT easy. I've never done it because you think about it while writing code. To be pedantic, all JavaScript functions block the event loop. It's just that the vast majority of functions execute so quickly that the amount of time your function blocks is very short. I once had a loop that processed tons of data and would block the event loop for 1-3 seconds. I ended up…

Right sorry. I meant for a significant amount of time that took down production or caused serious issues. I've taken down prod in other ways though :p

At one company I built API middleware that calculated time spent in the event loop vs waiting on dependencies (other apis, Redis, etc) by managing a couple counters per request. It was very helpful.

Re: We 30x'd our Node parallelism

#242

Post by Izkata »

MayeulC wrote: post #86
phoe-krk wrote: post #77

Earlier quoted context omitted.

An event loop is basically a single thread that executes functions from a FIFO queue. A function can put itself on a queue by "yielding", which means, allowing other functions to execute. If your function blocks, for example, by performing a wait on something without yielding, then nothing else gets computed because of the wait, but the event queue is occupied since the function has not yielded. This breaks the coope…

All right, thank you. So it's basically the same as any other non-preemptive multitasking design. If the event loop is part of the Node language/runtime, I can see the case for making it preemptive.

> So it's basically the same as any other non-preemptive multitasking design.

All this has happened before. All this will happen again.

Re: We 30x'd our Node parallelism

#243

Post by jrockway »

GordonS wrote: post #147
jrockway wrote: post #111

Earlier quoted context omitted.

Why? They had a 12 factor -ish app that scaled the normal way; run more copies. Eventually that got expensive. They had the observability to figure out what was making it expensive and whether or not their fixes had an effect. They then saved $300,000. Seems like everything went right to me. I would be worried if the blog post was "we randomly tweaked some stuff and we can't measure it but it's a little better" or "w…

> Why? They had a 12 factor -ish app that scaled the normal way So, yes, horizontal scaling is good, especially for stateless workloads - but that doesn't mean you run the most hopelessly under-performing code imaginable on each node, so you basically have to scale out like this! I mean, seriously, 4000 containers to serve 4000 concurrent requests? I mean, I can't even... I honestly can't believe the attempts in this…

I'm going to disagree.

When you start a business, you have no idea what it's going to grow into, or if it's going to grow. So you start simple. The design was good enough for there to one day be too many customers. That is huge.

When this happened, they started a second copy of their app, and could now handle twice as many customers. Repeat 3998 more times. Now the toy app is making some real money, so you can afford to deep-dive into the system and fix the technical problems.

They avoided the real issue that kills startups, having a customer call you because they want to buy your service and you saying "sorry, we aren't accepting any new customers right now because Hacker News comments don't like our software architectures."

Re: We 30x'd our Node parallelism

#244

Post by willglynn »

dmitriid wrote: post #211
coddle-hark wrote: post #53

Earlier quoted context omitted.

No it doesn’t. We’ve had good models for concurrency in single-threaded systems for a while now.

You say no it doesn't and then speak about concurrency models for single-threaded systems. Choose one :) Node can't not be single-threaded. Because Javascript is. Node.js is single threaded. It has a single event loop in a single thread, and all the "concurrency" is simply queued on that loop. It offloads some tasks to libuv for some system-related tasks but that's it. And the thread pool that libuv creates is very l…

Not OP, but… concurrency is not the same as parallelism. This is an important hair to split:

https://stackoverflow.com/a/24684037/1026671

"Concurrency" doesn't warrant scare quotes even when describing a single threaded program.

It's true that Node.js allows one task to block many others, but that's an implementation detail of Node.js, not a guaranteed result which necessarily applies to any program using only one OS thread. Other programs suffer from analogous starvation/deadlock/livelock/priority inversion problems, but those would be implementation details too, not guaranteed results from using multiple OS threads.

Re: We 30x'd our Node parallelism

#245

Post by jrockway »

tracker1 wrote: post #189
jrockway wrote: post #102

Earlier quoted context omitted.

The Node gRPC implementation is fine. It uses the C++ implementation which is the gold standard. It has Prometheus and OpenTracing interceptors. You basically give nothing up by using it, if your team wants to write a language that runs on node.

The bigger issue to me, is (at least the last time I looked) you can't use the cluster module with node combined with gRPC, so the only real way to take advantage of extra CPU capacity, if available is workers or external processes that are self-managed vs. cluster integration.

I think you can just run one node.js per core (or whatever the optimal balance is) and tell your load balancer that there are instances of your service available at hostname:8080, hostname:8081, etc. A lot of people are going to get this "for free" when they tell their container orchestrator that they want 8 replicas that each request 1 cpu, and the scheduler finds a node with 8 free cpus.

Re: We 30x'd our Node parallelism

#246

Post by [deleted] »

ubu7737 wrote: post #225
throwGuardian wrote: post #174

Earlier quoted context omitted.

> no way I'm going near Plaid with a very long bargepole after reading this But you'd go to a competitor who hasn't published a blog post, whose internal code you haven't audited and simply presume is just fine? In plaid's defense, lack of performance tuning isn't necessarily a lack of security focus.

ROFL "performance tuning" this is not tuning this is architecture. Some people think you can just write software, sell it to customers, and it's "tuning" to make it work properly. You should be fired from whatever job you have. My guess is that you have no job, you are fronting USD. In which case you have absolutely no place in this conversation and you should be ashamed of yourself for speaking up. A fool and his mo…

[deleted]

Re: We 30x'd our Node parallelism

#247

Post by todotask »

bjacokes wrote: post #216
danudey wrote: post #212

Earlier quoted context omitted.

I mean, my read of this is: 1. We used a system which uses event loops to achieve great concurrency, but we turned that off because we don't trust it. 2. Instead, we spent $300k/yr rolling out one-process-per-API as though we were using Apache 1.3. 3. We used an arbitrary JSON library without knowing anything about its performance characteristics, which it turns out were inordinately bad It's not that this wasn't a g…

I don't disagree with most of what you're saying. The Nth engineer at a startup rarely looks with admiration at decisions made by the (N/10)th engineer – but it was those decisions which helped the company grow to its current size. Likewise, I think most of us will be happy if the company 10x's again. Then some super-duper-senior engineer can look at the decisions we're making now – they're not perfect, but we're doi…

If I may add, Monzo bank which utilize Golang has interesting stack.

https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...

Re: We 30x'd our Node parallelism

#248

Post by neillyons »

eshyong wrote: post #46
phoe-krk wrote: post #24

Earlier quoted context omitted.

This is the worst kind of software engineering. There is a massive deadlocking design mistake in the centre of the language - literally a huge red button with DO NOT PRESS printed on it. Thousands of programmers pass it by every single day, or hour, or minute, and the creators of the runtime insist that it is impossible to fix that button whatsoever; instead, all users need to work around it by ensuring that their co…

I mean, running multiple node runtimes (aka multiprocessing) actually sounds like a reasonable compromise for parallelism. That's the standard solution for dynamic languages without great multithreading support. If you needed great multithreading support then Node probably wasn't the right choice for you in the first place, but for most applications, it's probably fine. However, running multiple containers for parall…

Yeah I’m confused why they didn’t start with running multiple processors per container. There must be a reason?

Re: We 30x'd our Node parallelism

#249

Post by coddle-hark »

dmitriid wrote: post #211
coddle-hark wrote: post #53

Earlier quoted context omitted.

No it doesn’t. We’ve had good models for concurrency in single-threaded systems for a while now.

You say no it doesn't and then speak about concurrency models for single-threaded systems. Choose one :) Node can't not be single-threaded. Because Javascript is. Node.js is single threaded. It has a single event loop in a single thread, and all the "concurrency" is simply queued on that loop. It offloads some tasks to libuv for some system-related tasks but that's it. And the thread pool that libuv creates is very l…

You’re focusing on single-threadedness like that’s a problem but it’s not. For example, you can run multiple processes on a single core system just fine — even if one of those processes hangs.

AFAIK there’s nothing in the JS language that forces a single-threaded implementation. I think the real reasons that Node is single threaded are a) legacy and more importantly b) a lot of existing code would break if events were dispatched in parallel.

Re: We 30x'd our Node parallelism

#250

Post by timmy-turner »

pdimitar wrote: post #150

...Or you could just use Erlang or Elixir, where concurrency and parallelism come pretty much out of the box, with very little effort required for you to fine-tune the desired policy / strategy. The insistence on using Javascript is just beyond lunacy at this point.

Well, if Elixir had a Typesystem like Javascript has, I'd instantly switch to it. But atm I'm staying with Node because of Typescript.
Post reply on HN