51–60 of 107 posts
Is there a way to run simulated traffic to determine how your server will react based upon heavier load to try and determine how many people it can serve?
[1] http://news.ycombinator.com/item?id=4847949 [2] http://www.joedog.org/siege-home/ [3] http://httpd.apache.org/docs/2.2/programs/ab.html
Running with swap enabled is a terrible idea. The authors mention how it was only once solr crashed that they were able to actually log in and start fixing problems; having swap means that rather than the OOM killer terminating processes, instead your whole system just grinds to a halt. (it's strange that they recommend enabling swap when they also recommend enabling reboot-on-oom, which is pretty much the complete o…
But modern wisdom on that is that, in general(+), it may be good to not have swap at all, on your server. Rather than address things by running parallel instances and load balancing.
As such swap space may also run out eventually if some service is leaking memory. And until it does it will make the system slow for everybody. Its better instead to let the culprit processes simply die, and make things easier for every body else.
On my server my jettys keep dying when they run out of memory. Thankfully there are lot other instances which are there to process requests.
Its a trade off you make, in favor of dropping some requests which are currently hitting the errant service (jetty instance in my case) vs. slowing down things for everybody, to the extent that even the developers can't help until something finally runs out of swap space also and dies (like the solr case you and OP mentions).
+ I say in general because there definitely could be reasons when you need swap.
Edit: Added explanation for (+)
If anyone owns a blog or site that they suspect may appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site. Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently. You should expect over 10,000 unique visitors within 24 hours, with most coming in the…
Oh, Well then let me share this. A few weeks ago I got on the front page and within a 24 hour period was hit with 29,000 unique visitors with 38,000 page views. The page itself is image heavy with 1.3 MB on first load. I'm running Wordpress with the Quick Cache plugin by PriMoThemes. I'm hosted on a shared 1and1 server. I've been hit before and went down, that's when I installed the Quick Cache plugin. Also 1and1 mov…
If anyone owns a blog or site that they suspect may appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site. Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently. You should expect over 10,000 unique visitors within 24 hours, with most coming in the…
Plug: we've built several products around Varnish so we have a good handle of how/where Varnish can be leveraged. Here's a list of varnish things we've built at unixy:
Varnish load balancer: http://www.unixy.net/advanced-hosting/varnish-load-balancer
Varnish for cPanel and DirectAdmin: http://www.unixy.net/varnish
Varnish w/ Nginx for cPanel: http://www.unixy.net/advanced-hosting/varnish-nginx-cpanel
Email in profile. I'll be more than happy to help out.
Joe
Is there a way to run simulated traffic to determine how your server will react based upon heavier load to try and determine how many people it can serve?
1. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection). 2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to 3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddo…
1. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection). 2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to 3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddo…
Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth.
You can use the optional second part the keepalive_timeout setting in nginx to send timeout hints to modern browsers, ie.
keepalive_timeout 10 10;
1. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection). 2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to 3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddo…
One must be careful when setting connection limits like this. A lot of people still use proxy servers and with modern browsers it quite easy to hit 20 concurrent connections per IP address.
Running everything on one box? Using swap? No caching? It's like a laundry list of junior admin mistakes.