Poor Man's Scalability
codypowell.com
Poor Man's Scalability
1–10 of 63 posts
Re: Poor Man's Scalability
#2Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".
Re: Poor Man's Scalability
#3With the rise of cloud services, and their relative inability to serve naked domains, there are a couple of easy fixes for this.
The first is to create a vhost in your webserver that serves an index.htm page on your NON-www url. The contents of that page would simply redirect users to the www.yourdomain url.
If that seems like too much work, an even easier fix (truncated for brevity) is to do something like this:
var url = window.location.href;
if(url.indexOf("http://www") != 0) {
window.location.replace("http://www.yourdomain/");
}
Re: Poor Man's Scalability
#4Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".
I'm very very reluctant to call those mistakes. From the author:
> I didn't get too upset here; I expected us to do poorly. After all, we'd spent all of our development time in actually building something people want, not on scaling up for thousands of imaginary users.
Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps they had to do consulting on the side for income.
Premature optimization is a mistake. It's time wasted on delivering minimal value to you or anyone else.
Re: Poor Man's Scalability
#5Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".
I'm very very reluctant to call those mistakes. From the author: > I didn't get too upset here; I expected us to do poorly. After all, we'd spent all of our development time in actually building something people want, not on scaling up for thousands of imaginary users. Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps t…
Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database.
Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the door." Your MVP isn't V if it still takes a second to render your homepage to users, because they'll leave. (Site responsiveness is a huge factor in bounce rate, even for sites that people actually want to look at.)
Re: Poor Man's Scalability
#6Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".
I'm very very reluctant to call those mistakes. From the author: > I didn't get too upset here; I expected us to do poorly. After all, we'd spent all of our development time in actually building something people want, not on scaling up for thousands of imaginary users. Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps t…
>Premature optimization is a mistake.
Wholly agreed and I often say the same thing. But I do not consider basic indexing on a database to be "premature optimization". I do consider it to be a a mistake since you almost certainly find performance issues (unless the table is small enough that indexing is a mistake), then you'll have to track down the cause of the issues (commonly called "a bug") and then will have to add indexes.
Re: Poor Man's Scalability
#7With the rise of cloud services, and their relative inability to serve naked domains, there are a couple of easy fixes for this. The first is to create a vhost in your webserver that serves an index.htm page on your NON-www url. The contents of that page would simply redirect users to the www.yourdomain url. If that seems like too much work, an even easier fix (truncated for brevity) is to do something like this: var…
Why not plain old HTTP 301/302 redirect?
Re: Poor Man's Scalability
#8Earlier quoted context omitted.
I'm very very reluctant to call those mistakes. From the author: > I didn't get too upset here; I expected us to do poorly. After all, we'd spent all of our development time in actually building something people want, not on scaling up for thousands of imaginary users. Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps t…
Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…
Indexes can most certainly be premature optimisation. There's a cost associated with an index.
Re: Poor Man's Scalability
#9Earlier quoted context omitted.
I'm very very reluctant to call those mistakes. From the author: > I didn't get too upset here; I expected us to do poorly. After all, we'd spent all of our development time in actually building something people want, not on scaling up for thousands of imaginary users. Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps t…
Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…
I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise.
First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differently, without needing that data. Then, finally, once you've used every trick to make it all faster, then you apply indices to speed up that handful of remaining queries.
If you apply indices first, then you won't spot the other potential optimisations, and when your indices stop covering up for your poor coding it will be much later, and much harder to fix.
Re: Poor Man's Scalability
#10It's fairly basic stuff this, but important none the less.
Kudos for recommending Yslow (or PageSpeed) - it's indeed brilliant. You didn't mention much about http caching, which I think is probably as important as the other things you mentioned; Not only does it improve performance for the user, but it also reduces load on your server and it enables you to put up an edge side cache for extra performance.
As for the problems mentioned with the load balancer - You could have simply provisioned a new instance and installed your own load balancer. There's dedicated packages like haproxy, but you could also just put up nginx or lighthttpd. This machine can later double as your edge cache (Squid).
Btw. 1.5s to render the front page? That's way beyond acceptable, in my book. But I suppose it depends a bit on what the web site does.