31–40 of 63 posts
This seems like basic stuff. I'm more interested in the story about how you got this to the top of the front page. Edit: Less obnoxious.
Earlier quoted context omitted.
> Btw. 1.5s to render the front page? That's way beyond acceptable, in my book. But I suppose it depends a bit on what the web site does. Agreed. This will have an extremely negative impact on your user engagement. Now if this is measuring end user response time rather than server-side page generation it might not be quite so bad, but generally I think most common pages should target < 150ms.
So what would you both consider "the next steps" in page speed optimizations?
With the rise of cloud services, and their relative inability to serve naked domains, there are a couple of easy fixes for this. The first is to create a vhost in your webserver that serves an index.htm page on your NON-www url. The contents of that page would simply redirect users to the www.yourdomain url. If that seems like too much work, an even easier fix (truncated for brevity) is to do something like this: var…
http://googlewebmastercentral.blogspot.com/2009/02/specify-y...
Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".
Oops, I think I didn't express myself very clearly. When I was sending 20 requests a sec in a load test, the average response time for all the requests was 9 seconds. In a normal case, the page was loading in a fraction of that. Also, we didn't lack all indexes, just a couple of important ones. We've been iterating quickly on the site and we weren't analyzing the performance as our queries were refactored.
We took a slow site running on hardware, put it on EC2 (slower hardware obviously), and with varnish are saving about 60% cost of hosting, and decreased page load time on some of the popular pages by upwards of 50%, all for minimal engineering effort.
Earlier quoted context omitted.
Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…
I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…
Earlier quoted context omitted.
I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…
I agree with you, but he was using MongoDB and AFAIK, indexing is a must with this kind of db. Actually, I hope that he does not expect a big write load because he seemed to have added the indexes pretty quickly without thinking too much about it... Just to reiterate, with a RDBMS with a decent query planner, indexing early is premature optimisation.
Earlier quoted context omitted.
I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…
No, caching should be the last trick you use to optimize. Indices are standard way to achieve basic performance levels of a database. They may have their downsides but "potentially dangerous" is dramatically overstating the case. Furthermore, the dangers of premature optimization are about taking extra time or adding complexity to something that ultimately doesn't matter, not about using very basic features in a sane…
You want to make the riskiest, most invasive changes first, because those are the ones that the rest of your codebase builds upon. If you've changed your query patterns and the app still isn't fast enough, it's relatively trivial to add indices on top of that. The speed benefits are cumulative, and none of the work you've done examining your data-access patterns has been undone by making that data access faster.
If you add indices first, however, and it still isn't fast enough, you have to examine your query patterns anyway. And this time, the work you did is undone by your further optimizations. The benefits of indices depend a lot on how you access your data: they slow down writes to speed up certain queries. If it turns out that the queries you're speeding up don't occur frequently anymore, your index is counterproductive.
(That said, since adding indices is often a task that takes 5 minutes of your time, you might want to give it a try and see what sort of results you get before investing a lot of effort in other stuff. If it doesn't work, you can back out the change and then start looking at your query patterns.)
Earlier quoted context omitted.
I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…
Ahem, this is so wrong it's not even funny. I'm just calling it out because there are people here who are just learning how to code. You always create an index, on pretty much every query. That's as basic as commenting your code.