21–30 of 49 posts
i thought node.js was fast as hell bad ass rock star tech that would never melt. http://www.youtube.com/watch?v=bzkRVzciAZg
well it's still pretty bad ass and rock star that you can measure server load by looking at "event loop lag", that's actually really neat... how do you do that with threads?
how should I know? :)
Great tip! Can you explain how you figured out the 200 request/sec limit?
https://gist.github.com/4532177#file-application_server-js-L...
Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous connections.
I think this would simplify implementing a load-balancing proxy in some ways. If a server responds with a 503 generated by node-toobusy, the load balancer will know immediately to reroute the request to another server, rather than having to wait for a timeout or some other threshold.
[edit] I do get your point, I was just saying: don't start implementing a naive load-balancer :-)
Interestingly, this is one piece of a broader strategy for building scalable web applications, addressed in Matt Welsh's thesis (SEDA):
http://www.eecs.harvard.edu/~mdw/proj/seda/
If you haven't taken a look at the ideas in SEDA, it's definitely worth the time. Most modern web apps incorporate at least some pieces of it to great effect.
Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous connections.
You'd do it because it's npm install toobusy then somewhere in your project require("toobusy"); If you only need one server that's going to be up to a lot faster than putting anything in front of it.
A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second grace), your first request will take the 5ms hit, but the next 199 for that second will be served from memory.
Now with Varnish serving 199 out of 200 requests from memory, if your backend is still toobusy, by all means serve a 503, and Varnish can cache that too.
Instead of returning a 503 page it could attach a ticket (by cookies) and say: "Your request will be solved in 55 seconds, be patient". The time calculation is based on statistics in the last 10 minutes or 1 hour...
The trick of using event loop lag is pretty neat, but the general strategy is IMHO a must have for any service. By aborting early and somewhat gracefully, the user knows ASAP, receives a controlled message that suggests she should not quickly retry, and the server can avoid trying to do part of the work for the request that is not going to be completed.