Live data from Hacker News

Poor Man's Scalability

codypowell.com

51–60 of 63 posts

Re: Poor Man's Scalability

#51
post #8

Earlier quoted context omitted.

Indexes can most certainly be premature optimisation. There's a cost associated with an index.

The cost increases significantly by doing it later on. Indexing when you have little data is easy. Indexing on a live site with millions of rows and lots of users can be much harder.

Taking it to that extreme is not a valid argument. There's a lot of room between not doing it first and only doing on a live site with millions of rows and lots of users.

Re: Poor Man's Scalability

#52
Cross posting my reply from another site:

> We were now rendering our site index in 1.5 seconds or less, compared to 3 seconds before.

This is the real WTF. What kind of queries take three seconds to run, that is ridiculous. Even with indexes 1.5 seconds is insane.

Deciding against premature optimisation isn't permission to throw away good, efficient design. It's clear from the rest of the article that these guys really have no idea what they're doing. Quoting the results of all their optimisations:

> In the end, we did not go down. The last round of load tests with httperf showed us handling 200 GETs a second indefinitely, with an average response time of about 1.2 seconds. We served thousands of requests without even a hiccup, and we did it without spending any money.

This is really poor performance, and it shows in the site when you change page or select a different category/filter. I'm guessing that they also chose a nosql solution for the wrong reasons, as it seems everything there could be done much faster/better by using an ACID compliant database.

Re: Poor Man's Scalability

#53
"Moving the Javascript to the bottom" - wouldn't the best thing to do would be to invoke the JS from onLoad? And put the code in a separate minified JS file, so it can be cached?

Re: Poor Man's Scalability

#54
post #8
post #5

Earlier quoted context omitted.

Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…

Indexes can most certainly be premature optimisation. There's a cost associated with an index.

That might be true for some esoteric indexes, but if we're talking about indexing customer_id on the invoices table, it's not premature -- it's freaking inevitable.

Re: Poor Man's Scalability

#55

"Moving the Javascript to the bottom" - wouldn't the best thing to do would be to invoke the JS from onLoad? And put the code in a separate minified JS file, so it can be cached?

Yes, but moving the actual script tag that loads that minified JS file to the bottom of the HTML can help in some situations. Less so now that most UAs have decent speculative parsers, but it used to be that seeing the script tag would block any resources below it from being requested until the script was done loading.

And being cached won't help for first-time visitors to the site, of course.

Re: Poor Man's Scalability

#56
post #21

Earlier quoted context omitted.

> Btw. 1.5s to render the front page? That's way beyond acceptable, in my book. But I suppose it depends a bit on what the web site does. Agreed. This will have an extremely negative impact on your user engagement. Now if this is measuring end user response time rather than server-side page generation it might not be quite so bad, but generally I think most common pages should target < 150ms.

So what would you both consider "the next steps" in page speed optimizations?

If you really do have a legitimate 1500 msec process occurring on the home page, try to flush as much of the HTML to the browser before you start the heavy lifting. Ideally, you could have the entire HTML shell delivered immediately, and the content elements slotted in at the very end (using DOM manipulation).

If you can put all the content inside a tag at the very end of the HTML, it will at least give the browser a chance to find and download all the page's resources (css, Jquery, logo image, etc etc).

e.g.

  
  
  
    Home Page
    cheap content

place-holder content

more content

~~flush~~ $("#content2").html("expensive content");

Re: Poor Man's Scalability

#57
post #8

Earlier quoted context omitted.

Indexes can most certainly be premature optimisation. There's a cost associated with an index.

That might be true for some esoteric indexes, but if we're talking about indexing customer_id on the invoices table, it's not premature -- it's freaking inevitable.

Yes--and these are the overwhelmingly most common case for a query in most applications. From reading the blog post, it seems like the blog author was either doing pretty frighteningly complex stuff just to render his homepage or he didn't think ahead enough to add fairly standard queryable indices to his collections.

Neither strikes me as terribly smart, and the latter strikes me as writing it right the first time, not "avoiding premature optimizations."

Re: Poor Man's Scalability

#58
post #25

Earlier quoted context omitted.

ooh, this looks really problematic on multiple levels (why wait for JS etc). A simple solution, which I've used--and I think the OPs situation is similar--is to just have an additional A-record pointed to the ELB, this should work unless the OP is doing something really fancy. No need for route-53, even go-daddy will let you do this. However, route-53 gives you much better TTL, so if you ever need to re-route..

ELBs change IP if they have to scale up. You really shouldn't point an A record at one.

...which is why they recommend you never use the ip, instead AWS provides a hostname, which is invariant...

Re: Poor Man's Scalability

#59
post #9
post #5

Earlier quoted context omitted.

Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…

I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…

That's just silly. Indexes are a fundamental component of databases and give the DB clues about how the data in your table is going to be used. You might as well saying having different column data types is premature optimization.

Should you spend days analyzing things and creating a million indexes? No. But you should have some idea how the tables will be used and setup a few of the obvious ones, at least. Read any book on DB admin.

Re: Poor Man's Scalability

#60
post #5

Earlier quoted context omitted.

Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database. Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the…

It's not hard to apply an index after the fact. You don't paint yourself into a corner by leaving them off initially. And very often, you can't understand your data model and how it will be requested until you've actually built an app and gathered some data. Users will surprise you, and do things you never expected, and probably render 80% of your app (oftentimes, the 80% you worked hardest on) obsolete immediately.

I can attest to that. We have two main "sides" of our app, one side that allows for editing of rules and another that just applies those rules on transactions. It took us much longer to build the editing side than than the transaction side, and when we hit users, we found out that they didn't even WANT to edit the rules. That was more than half of our app that users weren't using, even though in initial mockups/user tests they indicated that they would edit the rules.
Post reply on HN