Live data from Hacker News

Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

news.ycombinator.com

21–30 of 32 posts

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#21
post #20

My company did some email marketing that would spike traffic to 50,000 visitors in 30-60 mins and Wordpress MySQL would go nuts. Who knows how many horrible plugins marketing had in there but first go to for me was varnish, then cloud flare then final awesome fix was swapping everything to static HTML that we crawled every night ourselves from our own htaccess blocked Wordpress. Only thing that had to be dynamic /do…

On that same note I'm a huge advocate for static site generators. I build most of my sites anymore using Jekyll and just having forms submit to a php script or even just a Google Form.

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#22
If our current site got a 10x spike in traffic with little to no warning, the only thing we could do to prevent it from keeling over would be to raise the cache timeout on our caching proxy.

Why?

Well, we could easily add more application servers as needed. That would take about 15 minutes, and we'd probably only need to increase the count by about 50%, as we've got plenty of spare capacity.

The real problem is our database server. Or rather, the way our CMS uses the database. Any sort of traffic increase to the CMS hammers the DB to the point of deadlocks (yes, on reads) for other portions of the site. We have some plans to improve the situation (including changing some stupid DB config decisions made a decade ago), but nothing that can be implemented short term.

Thankfully, the CMS content is fairly static, and our 1-min caching isn't very aggressive. Increasing the cache timeout to 10-15 min would result in far less backend traffic and get us through most traffic spikes. The rest of our site either is available only to paying users (and thus far less likely to be significantly affected by a traffic spike) or served primarily by other data stores with much more room for capacity growth in their present configuration.

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#24
Static content in a ram disk and haproxy+apache replicas grown as required. No CDN.

Even the most "dynamic" sites in terms on non user-specific content can be turned into a static snapshot via a simple cron job. That means the dynamic content is only ever hit by one person, that being cron.

Real user-specific dynamic content must require a login and cookies for haproxy to even let the connection pass the first stage of validation.

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#25
If you don't make your website a megabyte-sized monster, it isn't a problem. Serving simple html/css content is possible to more users than hn has at any given point in time from a single-core Pentium 2. Anything more than that is just bloat.

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#26
I've had several posts on the front page running on small boxes without issue. At peak I saw 150-200 simultaneous users in Google Analytics. Sitting at the top of the front page for several days would probably see more than that. Putting a Cloudflare caching layer in front wouldn't hurt, but I didn't really need to do anything to prepare.

Alternatively just publish on a hosted blog service like WordPress, Medium, etc.

(This assumes your content is a post and not an app.)

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#27
post #23

My planned solution: Use Jekyll instead of WordPress. I have still not finished that migration.

Surely there are Wordpress plugins that handle caching/static page serving? Or is it insufficient?

Otherwise did you consider Grav[0] or Zotonic[1]

0. https://getgrav.org/

1. http://zotonic.com

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#28
post #20

My company did some email marketing that would spike traffic to 50,000 visitors in 30-60 mins and Wordpress MySQL would go nuts. Who knows how many horrible plugins marketing had in there but first go to for me was varnish, then cloud flare then final awesome fix was swapping everything to static HTML that we crawled every night ourselves from our own htaccess blocked Wordpress. Only thing that had to be dynamic /do…

>Who knows how many horrible plugins marketing had in there

Why aren't you proactively working with "marketing" to figure out why they installed the plugin and helping them reach their goal in another way?

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#29
Make everything static. Unless you really need some sort of dynamic content. If you do need dynamic content, make sure to stress test it. There are tools out there to load test your website to see if it breaks.

Put it on Cloudflare. Cloudflare can absorb huge volumes of traffic with ease. Keep in mind there are other WAFs (Web Application Firewalls) you can check out.

Use as little third party widgets / bells and whistles as possible, and self host assets when you can. If these go down (which they will when your site is trending on Hackernews), then your site may not load correctly and leave your users frustrated. Remember the recent S3 failure? It broke thousands upon thousands of sites.

Re: Ask HN: How would you handle a large traffic spike (eg. being frontpage on HN)?

#30
post #29

Make everything static. Unless you really need some sort of dynamic content. If you do need dynamic content, make sure to stress test it. There are tools out there to load test your website to see if it breaks. Put it on Cloudflare. Cloudflare can absorb huge volumes of traffic with ease. Keep in mind there are other WAFs (Web Application Firewalls) you can check out. Use as little third party widgets / bells and whi…

Also, if your site is dynamic but doesn't really need to update on every load you can set up Cloudflare to cache for a short period (e.g. 5 minutes).
Post reply on HN