Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poor Man's Scalability

codypowell.com

Post reply on HN
Print view View original

41–50 of 63 posts

Re: Poor Man's Scalability

#41

Post by _vn5r »

phadrus wrote: post #35
codypo wrote: post #28

Earlier quoted context omitted.

Oops, I think I didn't express myself very clearly. When I was sending 20 requests a sec in a load test, the average response time for all the requests was 9 seconds. In a normal case, the page was loading in a fraction of that. Also, we didn't lack all indexes, just a couple of important ones. We've been iterating quickly on the site and we weren't analyzing the performance as our queries were refactored.

Please stop blogging and start reading about how to construct an application properly.

+1

Re: Poor Man's Scalability

#42

Post by MichaelApproved »

tjr wrote: post #31
MichaelApproved wrote: post #14

This seems like basic stuff. I'm more interested in the story about how you got this to the top of the front page. Edit: Less obnoxious.

Basic stuff is valuable to people who need to learn the basics. Not everyone on HN is an experienced programmer, much less an experienced web programmer.

If you're a good enough programmer to know how to build the site, it's likely you know about profiling. Im surprised there are enough people who find themselves in this middle spot (can build complicated site but are still unaware of this info) to push the article up to the top page.

Having said that, nothing I submitted ever made it close to the top of HN. Maybe my understanding of the community is off.

Re: Poor Man's Scalability

#43

Post by MichaelApproved »

codypo wrote: post #29
MichaelApproved wrote: post #14

This seems like basic stuff. I'm more interested in the story about how you got this to the top of the front page. Edit: Less obnoxious.

Blackmail. I have pics of pg and Steve Ballmer sharing an ice cream cone, and I'll make them public unless I get the proper upvotes. I actually don't know what happened here either. I was at a kid's birthday party, and I look on Twitter to see one of my cofounders had submitted this and that it was shooting up the front page.

My tone was meant to imply there was some funny business going on but more in jest than reality. Clearly people are finding this article helpful.

Nice job.

Re: Poor Man's Scalability

#44

Post by mikescar »

nostrademons wrote: post #39
dasil003 wrote: post #13

Earlier quoted context omitted.

No, caching should be the last trick you use to optimize. Indices are standard way to achieve basic performance levels of a database. They may have their downsides but "potentially dangerous" is dramatically overstating the case. Furthermore, the dangers of premature optimization are about taking extra time or adding complexity to something that ultimately doesn't matter, not about using very basic features in a sane…

I would take swombat's advice to look at query patterns, eliminate repeated queries, and tweak your algorithm to require less data well before I thought about adding indices (or caching, which I agree is the last thing you should do). You want to make the riskiest, most invasive changes first, because those are the ones that the rest of your codebase builds upon. If you've changed your query patterns and the app stil…

Disagree on caching as the last thing you do.

If you're building an app that will hopefully be bigger than you expect, build cache into your data layer, it's not complex and will pay off sooner than later.

Re: Poor Man's Scalability

#45

Post by St-Clock »

St-Clock wrote: post #20
swombat wrote: post #9

Earlier quoted context omitted.

I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…

I agree with you, but he was using MongoDB and AFAIK, indexing is a must with this kind of db. Actually, I hope that he does not expect a big write load because he seemed to have added the indexes pretty quickly without thinking too much about it... Just to reiterate, with a RDBMS with a decent query planner, indexing early is premature optimisation.

Would someone care to explain the downvote?

Re: Poor Man's Scalability

#48

Post by ceejayoz »

jibs wrote: post #25
bmelton wrote: post #3

With the rise of cloud services, and their relative inability to serve naked domains, there are a couple of easy fixes for this. The first is to create a vhost in your webserver that serves an index.htm page on your NON-www url. The contents of that page would simply redirect users to the www.yourdomain url. If that seems like too much work, an even easier fix (truncated for brevity) is to do something like this: var…

ooh, this looks really problematic on multiple levels (why wait for JS etc). A simple solution, which I've used--and I think the OPs situation is similar--is to just have an additional A-record pointed to the ELB, this should work unless the OP is doing something really fancy. No need for route-53, even go-daddy will let you do this. However, route-53 gives you much better TTL, so if you ever need to re-route..

ELBs change IP if they have to scale up. You really shouldn't point an A record at one.

Re: Poor Man's Scalability

#49

Post by jacques_chester »

eytan wrote: post #11

Your story closely resembles the challenges we had scaling sommelierwine.ca before a media event at our latest launch location. I knew that our site would receive an increase in traffic, and the increase would only be temporary. As a rails site hosted on Heroku, this gave us the ability to scale our site using the gambit of widgets that Heroku offers; but, we didn’t want to spend the money and cheat ourselves from th…

> [1] This is database 101 stuff - a class I never took.

Yes, yes, a thousand times yes.

The best and most important way to improve database performance is to have a sensible schema. Query planners can do magic with stuff that's sensibly normalised and with things that are sensibly denormalised.

Where they choke is on schemata where everything is just squished into a bunch of tables with fields holding internal datastructures.

Re: Poor Man's Scalability

#50

Post by cageface »

dasil003 wrote: post #13
swombat wrote: post #9

Earlier quoted context omitted.

I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise. First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differ…

No, caching should be the last trick you use to optimize. Indices are standard way to achieve basic performance levels of a database. They may have their downsides but "potentially dangerous" is dramatically overstating the case. Furthermore, the dangers of premature optimization are about taking extra time or adding complexity to something that ultimately doesn't matter, not about using very basic features in a sane…

You should be thinking about indices as you design your data model. You're going to put them in eventually so the ad-hoc performance testing you're doing as you're building the site should at least somewhat reflect the final, real-world scenario. Better to know sooner rather than later that your data model is so broken that even extensive indexing can't make your queries fast.
Post reply on HN