31–40 of 51 posts
Earlier quoted context omitted.
Apache has not strictly been one process per connection since 2.0. See the worker MPM. I would have chosen nginx, but Apache can be configured as a capable static file server.
Even then each thread is still one connection (according to the apache docs).
Earlier quoted context omitted.
So I know nothing about haproxy, but what sort of system load does it generate? They say they ran it on a EC2 micro instance, and the micro instances are specifically designed to handle spikes, not continuous heavy load. In fact, they intentionally throttle under continuous heavy load. I guess it must have been the right choice for them, I was just surprised to see it was a micro. Would that typically be the right wa…
The micro instance was fronting the render farm. As far as I can tell the article doesn't specify the instance types in use for any of the other components.
Earlier quoted context omitted.
So I know nothing about haproxy, but what sort of system load does it generate? They say they ran it on a EC2 micro instance, and the micro instances are specifically designed to handle spikes, not continuous heavy load. In fact, they intentionally throttle under continuous heavy load. I guess it must have been the right choice for them, I was just surprised to see it was a micro. Would that typically be the right wa…
The micro instance was fronting the render farm. As far as I can tell the article doesn't specify the instance types in use for any of the other components.
I have always been a proponent of the 5 second cache for things like blog posts. After you hit Reddit, you can see 1000 requests a second for a few minutes. If you cache for 5 seconds, nothing is ever very far out of date, but you save yourself 5000 requests. Seems like a win-win. (I'm also a big fan of Varnish. I tried it out this weekend on a site that basically serves an HTML file that says "hello world". Apache c…
This is exactly why I created StaticGenerator for Django: https://github.com/luckythetourist/staticgenerator
I may sound naive here, but: if all you're doing is serving just 184 flat files, then why do you need all this RoR jazz? Can't a bunch of Apache/Nginx servers behind a load balancer (Varnish, TrafficServer, etc.) handle that just fine? Throw in a DNS-based scheme for failover/redundancy?
Forgive my ignorance, but does "flat file" mean "prerendered static HTML page" in this usage?
Sounds very much like it. From the article: >After each batch of new data was received from the AP, this server determined which pages needed to be re-rendered and, using the Typhoeus libcurl-multi bindings for Ruby, pulled new data for each of these pages from the render pool. Sounds like their infrastructure already has a farm of servers set up to handle rendering articles into their HTML components, so for this sc…
Earlier quoted context omitted.
Sounds very much like it. From the article: >After each batch of new data was received from the AP, this server determined which pages needed to be re-rendered and, using the Typhoeus libcurl-multi bindings for Ruby, pulled new data for each of these pages from the render pool. Sounds like their infrastructure already has a farm of servers set up to handle rendering articles into their HTML components, so for this sc…
All righty... it was just a usage of the term "flat file" I was unfamiliar with. :-)
A flat file is essentially a .csv holding data, and can be a fast way to bulk load denormalized data. As such, it actually does have a use in the context of scaling, so it's natural to expect that they were using the term correctly.
A static file, or more simple, a HTML file is what they're actually talking about. As they've noticed, it's what web servers are best at serving, and it scales obnoxiously well.
Now that we're all talking about the same thing, I can say that I've been serving all my product blogs as .html for years, and have never had any of them fall down under load.
It's really easy to get something in place to generate static files from a blog or CMS. I do it with:
- a 404 handler that maps missed requests for .html to their equivalent generator.
- a regular old blog engine that takes an extra parameter "writeThisToHTMLOnceYouveRenderedIt"
Future visitors skip the redirecting and generating and are simply served the static file. Next time you edit the content, you can simply blow away the whatever-blog-entry.html and index.html and know that they'll show up again next time anybody asks for one of them.
I have always been a proponent of the 5 second cache for things like blog posts. After you hit Reddit, you can see 1000 requests a second for a few minutes. If you cache for 5 seconds, nothing is ever very far out of date, but you save yourself 5000 requests. Seems like a win-win. (I'm also a big fan of Varnish. I tried it out this weekend on a site that basically serves an HTML file that says "hello world". Apache c…
How about cache it once at write-time, then again whenever you edit it?