Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surviving a traffic surge: Three techniques to scale your site fast

matt.might.net

Post reply on HN
Print view View original

1–10 of 44 posts

Re: Surviving a traffic surge: Three techniques to scale your site fast

#2

Post by bmelton »

Those tips are obviously worthwhile, but doesn't address massive scale for heavily dynamic sites. These should definitely be at the beginning of any optimization checklist for sure.

One thing I found interesting was his remark this:

Google Analytics failed to detect the surge: page load time was so high that visitors were closing the page before analytics could load.

He then remarks that it took Analytics 15 hours to detect the spike, but isn't that true of all Analytics instances? I'm not sure if the author is mistaking Google Analytics' delayed reporting as a fault or if I'm missing something.

Re: Surviving a traffic surge: Three techniques to scale your site fast

#4

Post by JoachimSchipper »

bmelton wrote: post #2

Those tips are obviously worthwhile, but doesn't address massive scale for heavily dynamic sites. These should definitely be at the beginning of any optimization checklist for sure. One thing I found interesting was his remark this: Google Analytics failed to detect the surge: page load time was so high that visitors were closing the page before analytics could load. He then remarks that it took Analytics 15 hours to…

These tips are for people getting slashdotted, not people building the next Google. Your average blog simply isn't "heavily dynamic", and if you're building Google/Facebook/... there are better resources.

Google Analytics is Javascript-based and thus vulnerable to people closing the page before it loads. It's actually near-realtime, but just doesn't work if you're having this problem. I'd imagine watching free Apache instances/threads, outgoing bandwidth or probably even the Apache logs would be more useful.

Re: Surviving a traffic surge: Three techniques to scale your site fast

#5

Post by rsloan »

bmelton wrote: post #2

Those tips are obviously worthwhile, but doesn't address massive scale for heavily dynamic sites. These should definitely be at the beginning of any optimization checklist for sure. One thing I found interesting was his remark this: Google Analytics failed to detect the surge: page load time was so high that visitors were closing the page before analytics could load. He then remarks that it took Analytics 15 hours to…

While the reason Google Analytics failed at this has already been explained, GA Ecommerce tracking is near real-time, but not error corrected. Data in other parts can take several hours to show up, Google says to wait 24 hours for data to be complete and error corrected.

Omniture SiteCatalyst is near-real time. I don't know if they do some error correction later like GA does for Ecommerce data.

Re: Surviving a traffic surge: Three techniques to scale your site fast

#6

Post by joelhaasnoot »

bmelton wrote: post #2

Those tips are obviously worthwhile, but doesn't address massive scale for heavily dynamic sites. These should definitely be at the beginning of any optimization checklist for sure. One thing I found interesting was his remark this: Google Analytics failed to detect the surge: page load time was so high that visitors were closing the page before analytics could load. He then remarks that it took Analytics 15 hours to…

GA data is delayed by a day, but if you adjust the date range manually, you can see the data that has come in so far. Not guaranteed to be complete, but some data.

Re: Surviving a traffic surge: Three techniques to scale your site fast

#7

Post by coderdude »

>>(Step 1: Cut image quality) Page load time dropped from 24 to 12 seconds.

Wow. I had no idea that could make such a difference. I suppose the issue was with the low number of threads set in the Apache configuration. The server was spending its time sending out static content when it could have been doing more important things? I signed up with S3 to serve up my static content to keep that load off my server. I should probably be using CloudFront instead though.

I'm interested in looking for a backup host just in case. I currently use WebFaction and I love them to death -- but I'm worried that under incredible stress the shared hosting won't hold. With Linode, do you start from scratch with a blank OS and just install everything you need from there (Apache, mod_wsgi, etc. kind of thing) or do they have preset installs? With WebFaction I can select a particular setup and I'm up and running in minutes.

Post reply on HN