Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node and Scaling in the Small vs Scaling in the Large

al3x.net

Post reply on HN
Print view View original

61–66 of 66 posts

Re: Node and Scaling in the Small vs Scaling in the Large

#61

Post by Retric »

cageface wrote: post #43
aaronblohowiak wrote: post #9

Earlier quoted context omitted.

As he notes, evented vs threaded is only important within a single machine. I would argue that your architecture for distributing load across machines is more important than evented vs threaded. If we think that the difference in performance is small (regardless of which is faster,) then the question becomes one of which programming style you prefer. Node's implementation of events means you can have mutable state wi…

I'm a little puzzled by this fascination with shared memory concurrency myself. If you have to scale beyond one physical instance, and you generally do if you have to scale at all, your scaling mechanism is your distribution mechanism. Of course, there are apps that need maximum throughput in a single address space but they seem to be very much in the minority.

That's like suggesting that you can eliminate CPU cache because most resources are going to be in main memory. Few systems scale 1:1 when adding more systems, yet making each system more powerful has the potential to be more useful than increasing the scaling factor.

In other words if 100 machines are 60 times more powerful than 1 machine, increasing your scaling factor is less important than doubling each machines throughput (assuming you keep the same scaling factor).

Re: Node and Scaling in the Small vs Scaling in the Large

#62

Post by rektide »

frognibble wrote: post #5
Tichy wrote: post #4

Earlier quoted context omitted.

It's just obvious how "threaded" can be slow even in low scale scenarios. It's less obvious how evented can be slower than the hardware allows (not saying it can't be). If nothing is known about the large scale, it still looks like a clear win for evented.

I keep hearing assertions that threaded can be slow or does not scale. Can somebody describe the reasons here or point to a good document on the subject? I often see "scalable" and "fast" used interchangeably in these discussions. These are different concepts. Is the contention that threading is not as scalable as evented, not as fast as evented, or both?

Threaded model has two detractors;

Foremost, context switching. This is a non-trivial process: on-die caches get flushed, virtual memory has to get re-mapped, registers re-loaded. This is by definition lost time, time the OS is spent not doing your workload. If the number of context switches gets too large, a system is spending an inordinate amount of time performing context switches, and less time doing "real work" inside threads. You cite two isues, slow and not scaling.

Second, memory usage. Each thread has its own stack space, which consumes a fixed memory size.

As to "slow" and "does not scale;" slow is relative; the amount of time spent context switching is very dependent on a variety of workload factors. Threading _can_ be slow. Scaling, on the other hand, is less unequivocal. A quarter million active threads, each doing very little, is not a good thing: too much memory, too many context switches, and too high a latency.

The problem is not threading per se. It's pre-emptive multi-threading, where a programs state is transparently pulled from the CPU by the OS and serialized to someplace else, and another programs state is de-serialized and loaded in transparent; that is, without the loaded or unloaded program knowing. Erlang and tasklets are examples of userspace or lightweighs threads where pre-emption is not permitted, and the context switching penalty is negligible. Instead of threads being ignorant, and being preemptively swapped in and out, they are coordinated and actively yield themselves back to the scheduler, thereby reducing the context switching penalty.

Re: Node and Scaling in the Small vs Scaling in the Large

#63

Post by rektide »

jhuckestein wrote: post #15

IMHO you may be able to scale node 'in the big' if you plan to scale on a level above node (i.e. in node instances and inter-node communication) instead of changing the code running in node to scale big.

ryah immediately posted the following series of tweets, to such effect:

"Thanks-some valid criticisms. The story doesn't stop at the process boundary. Concurrency through Spawning&IPC should be discussed"

"ie, "actors", although not explicitly given that name, are baked in. I just don't feel then need to put them inside the same process."

asked about impl discussion;

"Not really. It's not fancy: unix socks, no framing protocols, no RPC daemons. But using what's there is a valid concurrency solution."

Re: Node and Scaling in the Small vs Scaling in the Large

#64

Post by rektide »

jhuckestein wrote: post #59
rbranson wrote: post #57

What people tend to forget about Node.js is that everyone knows JavaScript. This is really it's killer feature against other platforms. The ability to get a team up and running on Node is unparalleled.

Unfortunately, everybody "thinks" they know JavaScript. This is dangerous in many ways and might just as well be a node disadvantage.

the particular danger i'd cite w/r/t JS is that its a much more flexible language than many, and that there's not the same level of entrenched best practices and tooling to enforce best practices. in other words, everyone knows JS, granted, but they know very different styles of it.

consider the plethora of means of doing inheritance that exist for JS.

Re: Node and Scaling in the Small vs Scaling in the Large

#65

Post by rektide »

felixge wrote: post #51

Node does support two models for concurrency: Non-blocking I/O and worker processes. Non-blocking I/O is no magic pony, so if you need access to more CPU cores, you start creating worker processes that communicate via unix sockets. I would argue that this is a superior scaling path than threads, because if you can already manage the coordination between shared-nothing processes, moving to multiple machines comes natu…

There are features you miss by relying on IPC. UNIX Sockets are zero copy, sure, but you still lose performance doing the marshal/unmarshal tasks, and that could be largely avoided using shared immutable memory.

I'd much rather node have a multi-threaded Web Workers implementation of nodejs that uses immutable shared memory for message passing. Not coincidentally, I'd much rather push the real distributed scaling problems (such as coordination and marshalling/demarshalling) up to a much much higher level in my application stack. As it stands now, I need multiple marshal/unmarshallers, for the IPC layer and the app's scaling-out/distributed layer.

Re: Node and Scaling in the Small vs Scaling in the Large

#66

Post by cageface »

Retric wrote: post #61
cageface wrote: post #43

Earlier quoted context omitted.

I'm a little puzzled by this fascination with shared memory concurrency myself. If you have to scale beyond one physical instance, and you generally do if you have to scale at all, your scaling mechanism is your distribution mechanism. Of course, there are apps that need maximum throughput in a single address space but they seem to be very much in the minority.

That's like suggesting that you can eliminate CPU cache because most resources are going to be in main memory. Few systems scale 1:1 when adding more systems, yet making each system more powerful has the potential to be more useful than increasing the scaling factor. In other words if 100 machines are 60 times more powerful than 1 machine, increasing your scaling factor is less important than doubling each machines t…

True, but a system that has to be parallel at the single, shared memory level and at the distributed level is going to be very, very complicated.
Post reply on HN