How do you scale your web application?

Measure and optimize. I write pretty careless code, as far as optimization goes, so first round is actually adding indexes to the database. I know what you want to say, I should do this from the start, right? But then I'd end up with a bunch of useless indexes who eat up update times and space.

Second step is usually cache - local, in-application cache. It's pretty much auto-pilot now, writing a cache function.

Cache helps, but not as much as you'd think. When it gets serious though it's still useful because its shape makes it easy to go to step three: mix the cache with a bit of refactoring. This can mean simply pre-fetching: there is a huge difference from doing 20 "select * from order_items where id=xxx" and just one "select * from order_items where order=yyy". And with the cache in place it usually means only a few extra lines of code. (This example is for building reports, not for displaying orders. I'm not that of an idiot to get each order_item separately).

If things get heavy refactoring can go as far as breaking the abstractions in place. Last month I squeezed an extra second by sending to the browser a big table in batches, while I was computing it. Not something I want to do every day, but occasionally it's fun.

This is also the step when I usually realize I made a stupid mistake along the way which caused the delay.

The only problem I met which was mostly impervious to this was a design decision. In one app I decided (as I usually do) to compute stuff like balances and debts on-the-fly. It usually (well, always) is a good idea, because any intermediary result is a source for bugs. But this particular app grew rather unexpectedly large, with the result that computing an overall balance takes 3-10 seconds. It's not a huge problem for the client, but my pride suffers. I'm still looking into this one - I think some refactoring may make it ok.

I usually find my apps are CPU bound rather than memory intensive, so it pays to test this out. Easy way to find out is to run apache bench locally and watch the server performance (Use perfmon on windows, not sure what the linux/unix alternative is). Its a crude performance metric but will point you in the right direction as to what you want to upgrade first, you can validate your findings with a more complex tool like Webload/JMeter/Wcat if you want to make sure.

Taken to its logical extreme, the cache becomes "your database," in which case you didn't need, for example, an RDBMS in the first place.

Max out the I/O capacity on your existing server:

At the very least, make sure each system bus (i.e. PCIe/PCI-X for commodity x86) slot is populated with a host adapter (ideally a hardware RAID controller) which can saturate that slot.

For each host adapter, ensure that each port has enough disks behind it that, even with worst-case contention (not fully-random, unless you've somehow managed to magically induce 512-byte transfers), they can saturate the the upstream system bus, even after RAID processing is done by the adapter.

After that, upgrade to a server with more slots and repeat the saturation.

Repeat once more for the largest single server available, while starting the project to implement (perhaps write from scratch) a distributed database.

I have yet to see anyone make even a concerted effort on the first step, instead skipping to the second part of the last step, probably because that appears, at first glance, to require only a Small Matter Of Programming.

The idea that everyone needs to scale horizontally has become very much less true than it was. Only Very Large sites have to go very horizontal at all these days.

Given 8 x 3Ghz core / 32GB / 5TB machines for cheap you're packing as much power as 15+ servers of a few years ago. Even quite large web sites can be run off a few beefy (but cheap!) servers. This means money saved on network infrastructure, power, space, management, etc.

Probably 80% of the Alexa top 1000 could each be run off less than 20 modern beefy machines.