Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Downsides of Caching

msol.io

Post reply on HN
Print view View original

31–36 of 36 posts

Re: Downsides of Caching

#31

Post by nstart »

makecheck wrote: post #3

Never implement a cache that you can't completely disable with the flip of a switch somewhere. Otherwise, when wrong behavior is observed in the system, you don't have an easy way to rule out the cached copies. And worse, as you make "changes" to the system code or the data, you may not be able to tell the difference between a change that had no effect on the problem and a change that was hidden by a stale cache. Thi…

How I handle kill switches is to first make sure that all code related to catching is just that. If it gets tied anywhere into the data saving logic or even worse, the actual business logic, you are screwed. I tend to manage this by keeping caching as part of a strategy pattern so I can enable or disable stuff using config parameters when starting the app. End to end testing with casper/selenium always runs with cache turned off unless I specifically want to test cache which I actually never have now that I think about it.

Re: Downsides of Caching

#32

Post by viraptor »

ggreer wrote: post #27
viraptor wrote: post #26

Earlier quoted context omitted.

I wouldn't say "never". Let's say you have a local file with 1e6 words. The file can be updated at some point. Your service gets a word in a request and needs to return "is this word in the list". Do you really want to read the file every time at request comes in? No, you're going to read it once and store it in an indexed set for quick lookup. You just cached a local data file. It's about the benefit vs. not caching…

Your example is quite valid, and I would probably implement something similar to solve the same problem. But it's not a cache. Caches can miss. Caches have a replacement policy. If it contains the complete, authoritative copy of the data, it's a memory-backed data store.

Re replacement policy - that's why I mentioned that the file can be changed. You'll need an mtime/inode/time check on each request / periodically.

Cache can miss? I don't think that's required. It can miss in a general sense, as in it needs to be lazy loaded. But I'd still call it caching if you're getting a single value. For example, you can still cache whole front page with server-side push into the cache. You also can't miss in that case.

But yeah, that's just details. Cache/memory structure is a rather vague separation.

Re: Downsides of Caching

#33

Post by derefr »

viraptor wrote: post #26
ggreer wrote: post #8

I agree with pretty much everything in this post, though I would add one more thing. It's not so much a downside of caching as a misuse: Application-level caches should never cache local data. Cache network responses. Cache the results of computations. Don't cache files or disk reads. Operating systems already implement disk caches, and they do a better job of it than you. That's in addition to a modern computer's nu…

I wouldn't say "never". Let's say you have a local file with 1e6 words. The file can be updated at some point. Your service gets a word in a request and needs to return "is this word in the list". Do you really want to read the file every time at request comes in? No, you're going to read it once and store it in an indexed set for quick lookup. You just cached a local data file. It's about the benefit vs. not caching…

I wouldn't call that a cache. It's certainly computed state, of the kind that can be thrown away and recomputed ala Crash-Only programming, but in my experience a "cache" is supposed to be transparent: to decorate an file/device/socket/RPC service/object/whatever and expose the same semantics as that whatever, but more performantly. Your indexed set doesn't expose the same semantics as the data file it was constructed from.

Re: Downsides of Caching

#34

Post by userbinator »

thrownaway2424 wrote: post #13
ggreer wrote: post #8

I agree with pretty much everything in this post, though I would add one more thing. It's not so much a downside of caching as a misuse: Application-level caches should never cache local data. Cache network responses. Cache the results of computations. Don't cache files or disk reads. Operating systems already implement disk caches, and they do a better job of it than you. That's in addition to a modern computer's nu…

I don't think that's very good advice in a heavily-loaded shared hosting environment. A disk read could easily stall for tens of seconds, just because the kernel whimsically decided to throw out the cache (or because your server crowded its memory container). I actually don't want any server touching a disk while it's serving. Everything should be read before service begins and never again.

In other words, you're advocating using more memory in a shared environment just so your server can (try to) be faster? What happens if everyone else does the same? Everyone ends up competing for the limited amount of memory, and no one wins.

"I'll grab all the memory I can so others can't use it" is a horrible way to think, as anyone who has attempted to simultaneously use multiple applications written with this mindset will know. One takes most of the memory, forcing other apps into swap, and then the opposite happens when you start working with one of the others, accompanied by massive swapping slowdowns.

Re: Downsides of Caching

#36

Post by velox_io »

Implement cache isn't hard conceptionally. If an object is modified, flush it from cache. If object isn't cached, build it.

The problem is that implementing caching is a bit of a canary in a coal mine. If there are problems with the architecture, then trying to add caching into the mix will make things much more difficult.

I wouldn't say adding cache to parts which you know will be heavily read, upfront (or at least adding hoods to make it easier to implement later) is a waste of time or "Premature Optimisation". The 80-20 rule is live and well, just use your judgement.

Post reply on HN