Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Your Site Hacker News Ready?

brandonsavage.net

Post reply on HN
Print view View original

21–30 of 51 posts

Re: Is Your Site Hacker News Ready?

#21

Post by jenius »

If you're hosting some sort of blog post, the best way to ensure that all goes well with high traffic is to make it static. Something like jekyll works great for this (publishing especially), and you can host any static site with a node server in like 3 lines of code on nodejitsu for $3/month.

Nodejitsu can handle hacker news traffic without even the slightest blink, with one dyno - I had a post up on the homepage for hours with over 200 concurrent visitors and like 13,000 total for the day and never had a single issue. Again, $3/month for that.

No need to over-complicate things : )

Re: Is Your Site Hacker News Ready?

#22

Post by AlexCP »

krapp wrote: post #15

Mostly as an educational exercise (perhaps in futility) I decided to try writing my own small php framework specifically to build and serve static files, load external libraries only when necessary and automatically send caching headers. The point being, for instance, not to even load Dwoo except if compiling the static pages (as opposed to fetching data from a view, building them and serving them with each request.)…

Building a static site generator is always a good learning exercise.

Re: Is Your Site Hacker News Ready?

#23

Post by brandonsavage »

eknkc wrote: post #17
brandonsavage wrote: post #16

Earlier quoted context omitted.

A misconfigured site that serves all its own assets will perform terribly because 200 hits per minute x 20 assets = 4,000 hits per minute. Wordpress has tons of assets too.

Never used WP, but aren't those static assets? The bottleneck would be bandwidth then, should not be a huge load on the server. If it's compiling stuff or fetching them from DB though, then that could cause some problems.

It has to do with the number of requests plus the number of allowed processes plus the requests each process was allowed to serve. At a limit of 300 requests per process, the processes fell over at about 3 seconds time. With only 20 processes, dying every three seconds, the server was overwhelmed. So it's really a combination of issues.

Looking at the records for MySQL data, there wasn't that much of an issue there with MySQL connections. This is probably due to the fact that the page cache was working correctly.

Re: Is Your Site Hacker News Ready?

#24

Post by brandonsavage »

eknkc wrote: post #14
Jabbles wrote: post #3

200 hits per minute? Is that all it takes? (I guess the answer is "No, only if you've misconfigured", but still...)

I was expecting a lot more that that given the number of servers going down as soon as they hit the front page.

I think that most people leave it configured for default and never bother to do any kind of caching. My server was down less than five minutes because I knew what to look for.

Re: Is Your Site Hacker News Ready?

#25

Post by danielhughes »

brandonsavage wrote: post #16
Jabbles wrote: post #3

200 hits per minute? Is that all it takes? (I guess the answer is "No, only if you've misconfigured", but still...)

A misconfigured site that serves all its own assets will perform terribly because 200 hits per minute x 20 assets = 4,000 hits per minute. Wordpress has tons of assets too.

The approach I've been taking with my recent projects is to serve all static assets from Amazon S3/CloudFront. That way 200 hits per minute is only 200 requests. 19 of those 20 assets in your example would get served from the CDN. I don't have a lot of experience building robust apps (I'm a self-taught hobbiest developer) but intuitively this configuration makes sense and it has been incredibly easy to implement. I did some stress testing for the first time on one of my apps using jmeter and was able to achieve much higher performance that 200 hits per minute on my free micro instance.

Re: Is Your Site Hacker News Ready?

#26

Post by Udo »

NginX + php-fpm + APC did the trick for me, at one time even in an EC2 Micro instance under heavy load.

There is a very nice hack I applied to my WordPress installs, but it's not for the faint hearted: hack the WP index.php to serve pages directly from Memcache for all non-authenticated GET requests. The beauty of this is that for most requests, the WP environment (which is horribly bloated) does not even get loaded.

It works about an order of magnitude better than all those cache plugins, but the downside is of course that software updates require some special care.

Re: Is Your Site Hacker News Ready?

#27

Post by danielhughes »

untog wrote: post #9
brazzy wrote: post #5

Earlier quoted context omitted.

If your site is entirely static content and mostly text, you don't really need a CDN. I've had a static site survive the HN frontpage without any problem on cheap-ass shared hosting.

If all you need is static hosting then S3 is great. The benefits of a CDN, and it's incredibly cheap too.

If you want the benefits of a CDN I think you'd have to link S3 to CloudFront. S3 alone does not provide CDN capabilities AFAIK.

Re: Is Your Site Hacker News Ready?

#28

Post by narcissus »

speeder wrote: post #20
narcissus wrote: post #13

Earlier quoted context omitted.

If you're just blogging, is there a reason you want to do it all yourself? I just set up a Blogger account with a custom domain and let it be...

I used to have a wordpress.com blog, but after they charged to everything I wanted to do, I got annoyed with them. Also, I like to sometimes fiddle with stuff, and not let my "webmastering" get too rusty. It is good to sometimes ssh to something that you own and wreck it at your will :) To me, part of the fun of setting up a blog, is not only writing, but also taking care of it. I don't know really why (I have no int…

That's fair. I have a similar thought process around other servers and sites that I have, so I can appreciate the need for fiddling.

Funnily enough, I think the reason I went with Blogger (alongside not needing to worry too much about traffic etc.) was specifically to take away my complete ability to fiddle... otherwise I tend to do all that instead of actually writing.

Re: Is Your Site Hacker News Ready?

#29

Post by thraxil »

lemcoe9 wrote: post #6
thraxil wrote: post #4

I'm constantly amazed at how poorly Wordpress is able to perform. 2.5GB virtual server with a cache in front of it and it went down? My blog is a Django app running on a 512MB Rackspace VM with a basic Apache, mod_wsgi, and Postgres setup. I have a dozen low traffic applications all running on the same server alongside it. I have zero caching set up so every hit is doing the full process of querying the database and…

I bet if you switched to nginx, your server would use even less resources to handle the same amount of traffic.

Having used nginx on other projects, I'm sure you're right. My blog setup is pretty old and hasn't really needed any improvements.

On newer deployments, I'm still using Apache+mod_wsgi to manage the Django processes (it's a setup that has been reliable for me and that I know how to debug), but I'm sticking an nginx proxy in front as a matter of course. Not for performance reasons, but just to leave myself a hook for handling later changes. I can trivially spin up additional application backend servers and load balance across them, or quickly flick a switch to have nginx cache a particular resource in an emergency.

Re: Is Your Site Hacker News Ready?

#30

Post by danso »

If you are self hosting Wordpress, the w3 Super Cache plugin is solid: http://wordpress.org/extend/plugins/w3-total-cache/

My otherwise poorly maintained blog didn't have a problem with a simultaneous HN/Slashdot hit, and I'm not even sure I'm using all the settings correctly. But the basic page cache seems to do what it promises.

Edit: oops, I see the OP is using Super Cache as well but still flopped. Then he is definitely doing someone wrong. My blog is hosted on the cheapest hosting plan that Dreamhost allows.

Post reply on HN