Live data from Hacker News

Sysadmin mistakes start-ups make

cloudkick.com

11–20 of 95 posts

Re: Sysadmin mistakes start-ups make

#11
post #4

If you fork inside an app server, such as mod_python, you will fork the entire parent process (apache!). This could happen by calling something like os.system("mv foo bar") from a python application. I nominate this post as the most distressingly important bit of information I've ever received at 2:43 AM in the morning. Now the question: what can I do in Ruby to avoid the four calls a second or so I'm currently makin…

[deleted]

Re: Sysadmin mistakes start-ups make

#12
post #6

I disagree with 1.3. "Serving static content is the easiest possible task for any web server." Yes, but keeping connections open for slow clients (esp with KeepAlive on) is not a good use of your 500MB Mongrel process' time. On the other hand, KeepAlive is a handy thing to have. Using a proxy like nginx or varnish to serve static files (and even dynamic data) if you have the proper KeepAlive and Nagle bits flipped ca…

Apache disables Nagle by default, which is what you want for small static files, but I'd love to see data showing that Nagle is actually a significant performance issue for a realistic load.

Re: Sysadmin mistakes start-ups make

#13
post #7
post #4

If you fork inside an app server, such as mod_python, you will fork the entire parent process (apache!). This could happen by calling something like os.system("mv foo bar") from a python application. I nominate this post as the most distressingly important bit of information I've ever received at 2:43 AM in the morning. Now the question: what can I do in Ruby to avoid the four calls a second or so I'm currently makin…

The solution to use an image processing library such as RMagick, http://rmagick.rubyforge.org/

Calling into RMagick/ImageMagick from inside the request/response cycle is probably even worse than shelling out, because ImageMagick does grievous damage to your runtime.

Re: Sysadmin mistakes start-ups make

#14
post #12
post #6

I disagree with 1.3. "Serving static content is the easiest possible task for any web server." Yes, but keeping connections open for slow clients (esp with KeepAlive on) is not a good use of your 500MB Mongrel process' time. On the other hand, KeepAlive is a handy thing to have. Using a proxy like nginx or varnish to serve static files (and even dynamic data) if you have the proper KeepAlive and Nagle bits flipped ca…

Apache disables Nagle by default, which is what you want for small static files, but I'd love to see data showing that Nagle is actually a significant performance issue for a realistic load.

You're right about Nagle; I mention it only because lighty or one of the others does not turn it off by default.

Having a lightweight proxy that keeps connections alive on the client end but cuts them off between themselves and the application layer is the bigger win all round for many real-world web loads.

Re: Sysadmin mistakes start-ups make

#15
post #10
post #4

If you fork inside an app server, such as mod_python, you will fork the entire parent process (apache!). This could happen by calling something like os.system("mv foo bar") from a python application. I nominate this post as the most distressingly important bit of information I've ever received at 2:43 AM in the morning. Now the question: what can I do in Ruby to avoid the four calls a second or so I'm currently makin…

I'd use something like DelayedJob and send_later the call to your image processing stuff, that way the forking happens out of the request path, at least.

You just described my exact setup. However, my understanding is that Delayed::Job's worker threads have a full Rails environment in them, and if this blog post is correct and I am indeed forking that entire Rails process for every call out to ImageMagick, my vague recollections of what a fork entails suggest to me that the Ghosts of C Programmers Past are going to visit a terrible vengeance upon me.

Re: Sysadmin mistakes start-ups make

#16
post #4

If you fork inside an app server, such as mod_python, you will fork the entire parent process (apache!). This could happen by calling something like os.system("mv foo bar") from a python application. I nominate this post as the most distressingly important bit of information I've ever received at 2:43 AM in the morning. Now the question: what can I do in Ruby to avoid the four calls a second or so I'm currently makin…

Four forks per second is basically nothing. This article is blowing it all out of proportion. You can't sustain forking per web request on a really large site but at this scale it's not going to matter.

The author is being stupid: the size of the process that you're forking doesn't really matter (it might start to matter if you didn't call exec() or exit() right after you forked, but that's not the case: you're just execing another program, which replaces the current process in memory). VERY little is copied; fork is defined to have copy-on-write semantics for the process's address space.

Re: Sysadmin mistakes start-ups make

#17
post #15
post #10

Earlier quoted context omitted.

I'd use something like DelayedJob and send_later the call to your image processing stuff, that way the forking happens out of the request path, at least.

You just described my exact setup. However, my understanding is that Delayed::Job's worker threads have a full Rails environment in them, and if this blog post is correct and I am indeed forking that entire Rails process for every call out to ImageMagick, my vague recollections of what a fork entails suggest to me that the Ghosts of C Programmers Past are going to visit a terrible vengeance upon me.

Only if you run out of memory. But with DJ at least you should be forking only one call at a time, rather than multiple, like you might from the controller itself. So although you'll end up using more memory, it'll only be one extra rails process, not 4.

May not still be ideal... interested to hear other people's ideas.

Re: Sysadmin mistakes start-ups make

#18
post #6

I disagree with 1.3. "Serving static content is the easiest possible task for any web server." Yes, but keeping connections open for slow clients (esp with KeepAlive on) is not a good use of your 500MB Mongrel process' time. On the other hand, KeepAlive is a handy thing to have. Using a proxy like nginx or varnish to serve static files (and even dynamic data) if you have the proper KeepAlive and Nagle bits flipped ca…

It's almost always a bad idea to use anything other than a non-blocking/async server to handle static content.

I think it's simpler/easier (maybe faster) to serve content from a separate sub-domain (static.site.com or whatever). Using a reverse proxy works too, but unless you're caching dynamic content it's probably no benefit and it's less efficient.

Re: Sysadmin mistakes start-ups make

#20
post #6

I disagree with 1.3. "Serving static content is the easiest possible task for any web server." Yes, but keeping connections open for slow clients (esp with KeepAlive on) is not a good use of your 500MB Mongrel process' time. On the other hand, KeepAlive is a handy thing to have. Using a proxy like nginx or varnish to serve static files (and even dynamic data) if you have the proper KeepAlive and Nagle bits flipped ca…

The default KeepAliveTimeout setting for Apache is 15 seconds, which is too long. Many of our large customers are setting KeepAliveTimeout to 2 seconds which frees up that apache worker to process new requests fairly quickly. You'd be surprised how many people never change this value from the default.
Post reply on HN