Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building A Node.js Server That Won't Melt

hacks.mozilla.org

Post reply on HN
Print view View original

41–49 of 49 posts

Re: Building A Node.js Server That Won't Melt

#41

Post by toast0 »

jconley wrote: post #15

This is great to see! IMO it's something that should be configurable and built into http in node.js. IIS [1] and other mature web servers like nginx [2] or apache [3] do this kind of thing for you and provide simple configuration on it. [1] http://support.microsoft.com/kb/943891 [2] http://serverfault.com/questions/412323/nginx-503-error-in-h... [3] http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxrequ...

Apache's MaxRequestsPerChild directive is not at all about limiting load; it says after serving N requests (or N connections in a keep-alive setting), the worker kills itself and another may be spawned in its place (subject to spare server/thread config). This mostly helps keep slow memory (or other resource) leaks in check by starting fresh every so often.

Did you mean to link to MaxClients[1]? MaxClients sets the maximum number of simultaneous connections; any additional connections will be queued by the OS socket api (subject to listen backlog, etc).

I think waiting to accept sockets that you can't handle is a better solution than either accepting a socket to return an error message, or (much worse) accepting a socket that overloads your system. Unfortunately, sometimes it can be hard to set MaxClients to the right value that isn't so big that you get reduced througput, or too small that you don't use all your resources. (One thing that does help you get to the right number for MaxClients is to set MinSpareServers to the same value as MaxClients; you will avoid issues where MaxClients is too big and you start swapping during high load, but you don't notice it because things are fine with a small number of servers).

Sorry, that was way too much information.

[1] http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclie...

Re: Building A Node.js Server That Won't Melt

#42

Post by politician »

It'd be interesting to experiment with pairing this to a proof-of-work system (e.g. Hashcash) to smooth the transition between toobusy and !toobusy. With an additional signalling mechanic, it could help the surge against peers in a cluster when one server begins to struggle.

Though you'd need a custom client, so maybe for appcache'd webapps or mobile apps, but not typical webapps. Or browser vendor participation (not serious (ok, half serious)).

Re: Building A Node.js Server That Won't Melt

#43

Post by charliesome »

taf2 wrote: post #18
tbatterii wrote: post #17

i thought node.js was fast as hell bad ass rock star tech that would never melt. http://www.youtube.com/watch?v=bzkRVzciAZg

well it's still pretty bad ass and rock star that you can measure server load by looking at "event loop lag", that's actually really neat... how do you do that with threads?

by using system load averages

Re: Building A Node.js Server That Won't Melt

#44

Post by benologist »

byoung2 wrote: post #29
benologist wrote: post #6

Earlier quoted context omitted.

You'd do it because it's npm install toobusy then somewhere in your project require("toobusy"); If you only need one server that's going to be up to a lot faster than putting anything in front of it.

If you only need one server that's going to be up to a lot faster than putting anything in front of it. A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second…

That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache.

  var cache;

  module.exports = function(request, response) {
    if(cache) {
      return response.end(cache); 
    }

    // get my data from wherever
    cache = the_data;

    return response.end(cache);
  }
Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of the language instead of a 3rd party tool you have to select and configure.

Re: Building A Node.js Server That Won't Melt

#45

Post by chetanahuja »

This is a neat little technique and the fact that the developers did this puts nodes.js higher in my stack of "technologies to consider".

A harder problem is to solve the same "melt-out" problem for the entire gamut of servers that are usually found in large serving stacks (various web servers, relattional and nosql db servers etc). The users are usually not the developers of these pieces of technology and there's never enough dev-bandwidth available to actually implement self-tuning techniques like the one mentioned here.

For such situations I once came up with a "little" trick to limit the damage in sudden overload situation. It allows the admins of said servers to tune the threadpool and request queue lengths in an intelligent way (with the help of historical performance data that most shops should have). I mentioned it in this discussion thread here: http://www.linkedin.com/groups/Whats-generally-used-methodol...

Re: Building A Node.js Server That Won't Melt

#46

Post by byoung2 »

benologist wrote: post #44
byoung2 wrote: post #29

Earlier quoted context omitted.

If you only need one server that's going to be up to a lot faster than putting anything in front of it. A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second…

That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache. var cache; module.exports = function(request, response) { if(cache) { return response.end(cache); } // get my data from wherever cache = the_data; return response.end(cache); } Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of th…

How do you selectively serve the cached response to some users based on cookies, header, etc., and expire it after a set amount of time? How would you gather stats about how many cache hits vs misses you have? How do you serve the cached response when your server is pinned? These are problems 3rd party tools have solved.

Re: Building A Node.js Server That Won't Melt

#47

Post by benologist »

byoung2 wrote: post #46
benologist wrote: post #44

Earlier quoted context omitted.

That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache. var cache; module.exports = function(request, response) { if(cache) { return response.end(cache); } // get my data from wherever cache = the_data; return response.end(cache); } Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of th…

How do you selectively serve the cached response to some users based on cookies, header, etc., and expire it after a set amount of time? How would you gather stats about how many cache hits vs misses you have? How do you serve the cached response when your server is pinned? These are problems 3rd party tools have solved.

You can expire things with setInterval and a counter. You serve a cached response if your server has a cached response, or if you are using it as a fallback you would combine it with something like toobusy maybe so you have a functional (if not fresh) 'under load' page.

Where it gets really fun with NodeJS is you can do all of your data work outside of the requests so you can pull all of your content out at the start and refresh it on an interval independently of the users, which can eliminate some or all of their trips to the database if you're lucky and it fits in ram and is viable etc.

The less moving parts that need to cooperate to serve your site the better - most things don't warrant a deep stack of technology to serve HTML and run CRUD operations.

Re: Building A Node.js Server That Won't Melt

#48

Post by nakkiel »

j45 wrote: post #37

Forgive my ignorance, but I thought the whole point of Node.js was not to meltdown. Are there some recommend resources I could learn more from about the out of the box scaling vs tactics and strategies that benefit growth? I love learning how this problem is tackled anywhere. :)

There's only so much load any piece of software running on a given and finished set of resources can handle properly... Once limits are reached, things will misbehave.

Re: Building A Node.js Server That Won't Melt

#49

Post by j45 »

nakkiel wrote: post #48
j45 wrote: post #37

Forgive my ignorance, but I thought the whole point of Node.js was not to meltdown. Are there some recommend resources I could learn more from about the out of the box scaling vs tactics and strategies that benefit growth? I love learning how this problem is tackled anywhere. :)

There's only so much load any piece of software running on a given and finished set of resources can handle properly... Once limits are reached, things will misbehave.

I agree completely, have lots of experience in complex hosting.

With all software having it's limits (assuming hw is relative) has a starting and ending point of when you need to start tweaking it, and in what direction. I was hoping to read more about that.

Post reply on HN