How Google Taught Me to Cache and Cash-In • Hacker News Board

Re: How Google Taught Me to Cache and Cash-In

#21

Post by peoplerock » Sun, Sep 13, 2009, 7:01 PM UTC

desu wrote: post #15

jules wrote: post #12

Earlier quoted context omitted.

You could load the user's votes into a cookie.

It would get too large quickly. Check this out for a good explanation of why you want to keep cookie sizes as small as possible: http://yuiblog.com/blog/2007/03/01/performance-research-part...

Thanks for that link. Interesting on the how ebay and myspace use cookie size with great abandon (v. Amazon & Google, for ex.). Leaves me wondering whether they know what it's costing them. And if so - whether they are realizing significant savings with other performance figures.

Re: How Google Taught Me to Cache and Cash-In

#22

Post by jasonkester » Sun, Sep 13, 2009, 10:12 PM UTC

alexandros wrote: post #3

I wonder if anyone's gone the next step and written updates to the caches directly, leaving out the database write from the critical path.

We do this for the Twiddla sandboxes.

The key is that you have to be prepared for everything to evaporate if the server cycles its memory for whatever reason. In this specific case, we toss everything out of the sandboxes every 5 minutes & reset them anyway, so it's no great tragedy if they clean up after themselves from time to time.

For something with a ton of low-priority edits (like HN or Reddit where nobody sues you if their vote goes away), you could certainly get away with caching updates and only saving them out every once in a while.

Re: How Google Taught Me to Cache and Cash-In

#23

Post by jasonkester » Sun, Sep 13, 2009, 10:18 PM UTC

One extra point: Don't Add Caching Yet.

1. wait until something presents itself as a bottleneck

2. optimize it until it's not a bottleneck anymore

3. wait until it presents itself as a bottleneck again anyway.

...Then add caching.

You'll never find the low-hanging yet dog-slow fruit if you put your aggressive caching scheme in place right off the bat. If you have a bunch of poorly optimized code running with a ton of caching to hide it from you and you suddenly have enough traffic to cause scaling pain, that's a big problem.

Speed it up. Then cache it.

Re: How Google Taught Me to Cache and Cash-In

#24

Post by alexandros » Mon, Sep 14, 2009, 11:18 AM UTC

laut wrote: post #19

alexandros wrote: post #3

I wonder if anyone's gone the next step and written updates to the caches directly, leaving out the database write from the critical path.

Do you mean that when you get an update, you both write to the database and perform an update in the cache? If so, you would want it to be a transaction so that the cache and database is not out of sync. Maybe a write cache, like on hard drives with write back caching. If you first wrote the update to a cache (queue of updates). Then it would be written to both the cache (cached html for instance) and database in one…

transactions are unnecessary, unless you are looking for 100% consistency between cache and database at all times. Usually this is not the case and if it is, you will have extremely serious problems scaling. All i'm saying is: If you have an update that you know will invalidate some cached pages, why invalidate them and then rerender them from the database once the update is commited instead of updating them directly. and lazy queueing the update, taking the load of the database (less pressure to commit instantly, and less reads). In the facebook inbox example, you know that a new message to the user will increase their message count by one. Just update the cache directly with this new info. When/if it explodes, by all means, rerender everything.

Re: How Google Taught Me to Cache and Cash-In

#25

Post by bmelton » Mon, Sep 14, 2009, 12:28 PM UTC

desu wrote: post #7

bmelton wrote: post #5

Earlier quoted context omitted.

In the past my strategy was basically that. Load a static page from cache wherever possible. If the user performs a vote, or submit, or what have you, write it to the database, but just perform the update on the user's local page in the DOM.

Yeah, that's what you'd do as the user makes a vote. But how would you load the page in the first place? The front page is different for each user, as they've likely voted up and down numerous links on it. You could load a generic page and then apply their votes by AJAX or something but that doesn't strike me as being any more efficient for the DB than just generating the page.

For the most part, we were serving a generic page, except for prior actions done by the user. The only major differences between your page and my page, for example, is that article 5 might not show vote links at all, if I had previously voted on them.

For the most part, we accomplished this through cookies. If the cookie exists, read in a hashmap of activities you've recently performed, and update the page DOM based on those. If the cookie didn't exist, then go ahead and do a full query from the db, repop the cache and build the cookie.

That said, we noticed that in a lot of cases, that wasn't ideal, and we started rendering partial templates -- so instead of loading a whole page from cache, we would load page parts, basically one part for each article. This worked well, as each template was basically a pre-rendered static file, and we only had two basic states to work with -- whether the user had voted on this before or not.

Our new strategy then was to query just the indexed table for a user's recent votes, separately query an index for which articles to display, and load the templates for the to-be-displayed articles based upon the user's vote state.

That allowed the page to render a lot faster, but kept the box under more load. Though we suspect it would have scaled well enough (had the project not eventually fizzled out), it wasn't ideal server-side, though delivered a vastly superior end-user experience.

YMMV.