Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node and Scaling in the Small vs Scaling in the Large

al3x.net

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Node and Scaling in the Small vs Scaling in the Large

#51

Post by felixge »

Node does support two models for concurrency: Non-blocking I/O and worker processes.

Non-blocking I/O is no magic pony, so if you need access to more CPU cores, you start creating worker processes that communicate via unix sockets. I would argue that this is a superior scaling path than threads, because if you can already manage the coordination between shared-nothing processes, moving to multiple machines comes natural.

Otherwise I agree with the post. Nothing will allow you to scale a huge system "easily".

Re: Node and Scaling in the Small vs Scaling in the Large

#52

Post by weixiyen »

jerf wrote: post #26
Tichy wrote: post #22

Earlier quoted context omitted.

I am not a specialist on this area. But please mention some interesting competitors to Node.js for your type 2? Personally I also like the fact that Node is Javascript. I have looked at Scala several times and superficially, it looks extremely ugly. Clojure and Haskell might be interesting, but I worry that the non-modifiable memory could bite me in the end (plus, Clojure might force me to do too much dreaded Java st…

You mentioned most of the ones I consider the real winners. Go is another possibility, though I'm not sure it fits in here; I don't know what sits at the core of the goroutines scheduler, but if it isn't select-like it probably easily could be. There are also a metric shitload of other event-based frameworks like Node.js, only for other languages, at varying levels of maturity and cruftiness. (Though part of the reas…

> Because twice in my experience I have started on one of these bases and managed to blow out their complexity budget as a single programmer working on a project for less than a year.

Could you please share a little more about this? I'm interested to know what it is you were building solo and the tools you were using, and what made it so difficult to maintain.

Re: Node and Scaling in the Small vs Scaling in the Large

#53

Post by draegtun »

jerf wrote: post #26
Tichy wrote: post #22

Earlier quoted context omitted.

I am not a specialist on this area. But please mention some interesting competitors to Node.js for your type 2? Personally I also like the fact that Node is Javascript. I have looked at Scala several times and superficially, it looks extremely ugly. Clojure and Haskell might be interesting, but I worry that the non-modifiable memory could bite me in the end (plus, Clojure might force me to do too much dreaded Java st…

You mentioned most of the ones I consider the real winners. Go is another possibility, though I'm not sure it fits in here; I don't know what sits at the core of the goroutines scheduler, but if it isn't select-like it probably easily could be. There are also a metric shitload of other event-based frameworks like Node.js, only for other languages, at varying levels of maturity and cruftiness. (Though part of the reas…

Not tried POE myself but it does seem to produce a wide love/hate divide among its users. AnyEvent (http://search.cpan.org/dist/AnyEvent/) seems to be the more pragmatic choice.

Re: Node and Scaling in the Small vs Scaling in the Large

#54

Post by cgbystrom »

moe wrote: post #14

(No, I’m not making the “callbacks turn into a pile of spaghetti code” argument, although I think you hear that time and again because it’s an actual developer pain point in async systems.) Amen. On my first try I was mostly underwhelmed with node precisely because of the callback hell you end up in. I've already had my share of that in twisted and despite the arguments various people make for it; thanks, but no than…

This the exact reason why I dumped Twisted in favor for gevent (greenlet/coroutine-based NIO library)

Re: Node and Scaling in the Small vs Scaling in the Large

#55

Post by cgbystrom »

I think Alex confuses "scaling in the small" with performance. Rewriting Starling in Scala most probably improved Twitter but it did it make it more scalable?

To me as an outsider, it sounds more as Starling (or Kestrel) got its performance upgraded, not scalability.

May sound pedantic, but it's important to differentiate performance from scalability.

Re: Node and Scaling in the Small vs Scaling in the Large

#56

Post by _urga »

blasdel wrote: post #12

He mostly pans the 'less-than-expert programmers' canard, so he never explores the assertion that's at its base — that events are far better for correctness . Rob Pike has spent the last 25 years developing evented languages, and while the Hoare-style CSP approach he's settled on allows for physical concurrency, he doesn't give a shit about bare-metal performance. The fundamental purpose is to be able to write concis…

"to be able to write concise programs that directly model the parallelism of the real world"

You nailed it.

Re: Node and Scaling in the Small vs Scaling in the Large

#58

Post by swannodette »

Tichy wrote: post #22
jerf wrote: post #11

Earlier quoted context omitted.

Terminology is part of the problem here. Comparing "threaded" vs. "evented" may have made sense in, say, Windows in 2003 or some context where we are clearly talking "select loop" vs. "pthreads", but there are a lot of things that could be called "threading" models now. Well, sort of a lot of things, because they're all converging now on the same core primitive, leaving us with two basic cases here: 1. Old, synchrono…

I am not a specialist on this area. But please mention some interesting competitors to Node.js for your type 2? Personally I also like the fact that Node is Javascript. I have looked at Scala several times and superficially, it looks extremely ugly. Clojure and Haskell might be interesting, but I worry that the non-modifiable memory could bite me in the end (plus, Clojure might force me to do too much dreaded Java st…

Clojure 1.2.0 lets you create data structures with mutable fields in pure Clojure if you really need the perf and you know what you are doing.

Re: Node and Scaling in the Small vs Scaling in the Large

#59

Post by jhuckestein »

rbranson wrote: post #57

What people tend to forget about Node.js is that everyone knows JavaScript. This is really it's killer feature against other platforms. The ability to get a team up and running on Node is unparalleled.

Unfortunately, everybody "thinks" they know JavaScript. This is dangerous in many ways and might just as well be a node disadvantage.

Re: Node and Scaling in the Small vs Scaling in the Large

#60

Post by technomancy »

Tichy wrote: post #22
jerf wrote: post #11

Earlier quoted context omitted.

Terminology is part of the problem here. Comparing "threaded" vs. "evented" may have made sense in, say, Windows in 2003 or some context where we are clearly talking "select loop" vs. "pthreads", but there are a lot of things that could be called "threading" models now. Well, sort of a lot of things, because they're all converging now on the same core primitive, leaving us with two basic cases here: 1. Old, synchrono…

I am not a specialist on this area. But please mention some interesting competitors to Node.js for your type 2? Personally I also like the fact that Node is Javascript. I have looked at Scala several times and superficially, it looks extremely ugly. Clojure and Haskell might be interesting, but I worry that the non-modifiable memory could bite me in the end (plus, Clojure might force me to do too much dreaded Java st…

> Clojure and Haskell might be interesting, but I worry that the non-modifiable memory could bite me in the end

For the record, this is absolutely no problem given the generational GC in the Hotspot JVM; the structural sharing in Clojure's persistent data structures generate a little extra garbage, but it's barely a blip on the radar. Laziness can have more of an impact on memory usage, but it's usually really easy to spot.

Post reply on HN