Downsides of Caching
msol.io
Downsides of Caching
1–10 of 36 posts
Re: Downsides of Caching
#2I agree with all the main points here: caching adds a significantly complex component to the system. You should only do it if you absolutely must pull data closer to a consumer. Adding caching "to pick up quick wins" is always dumb.
With that in mind, I do think most of the pitfalls listed here can be avoided with well-understood tools and techniques. There's no real need to be running your cache in-process with your GC'd implementation language. Cache refilling can be a complex challenge for large scale sites, but I expect that a majority of systems can live with slower responses while the cache refills organically from traffic.
The points about testing and reproducible behavior are dead on - no equivocation needed there. As always keeping it as simple as possible should be a priority.
Re: Downsides of Caching
#3Never implement a cache that you can't completely disable with the flip of a switch somewhere. Otherwise, when wrong behavior is observed in the system, you don't have an easy way to rule out the cached copies. And worse, as you make "changes" to the system code or the data, you may not be able to tell the difference between a change that had no effect on the problem and a change that was hidden by a stale cache.
This is one of the things that drives me crazy about some of Apple's technologies. For instance, somebody at Apple decided long ago that all application HTML help pages should be cached. The "off switch" for this cache remains a bit of black magic but it's something like "rm -Rf com.apple.help.DamnedNearEverything" followed by "killall UnnecessaryBackgroundHelpProcess" every damned time you modify a page or else the help system might show you an older version of the content that you just "changed".
Re: Downsides of Caching
#4Caching is also bad in distributed systems, because by definition you're creating tail latency: the cache miss case. In a distributed system, you're more likely to hit the worst case in one component, so the cache may not buy you any end user benefit. It might just make performance more difficult to debug.
A cache can still be useful if to reduce load and increase capacity... but latency becomes more complex.
Re: Downsides of Caching
#5Great article. I'll add one: caching doesn't address underlying performance issues, just "sweeps them under the rug".
I've seen many devs jump to caching before investing time in understanding what is really causing performance problems (I was one of them for a time, of course). Modern web stacks can scream without any caching at all.
Years ago, a talk by Rasmus Lerdorf really opened my eyes up to this idea. [1] He takes a vanilla PHP app (Wordpress, I think) and dramatically increases its throughput by identifying and tweaking a few performance bottlenecks like slow SSL connections. One of the best lines: "Real Performance is Architecture Driven"
[1] I think it was a variation of this one: https://vimeo.com/13768954
Re: Downsides of Caching
#6Caching is the classic memory-time tradeoff, everything from memoizing a function, to DNS caching, to storing a web resource that didn't change.
I think that, if a cache is combined with a push indicating a change, then it's basically a local "eventually consistent replica" which catches up as soon as there is a connection to the source of truth.
Seriously, many times you are READING data which changes rarely (read: every X minutes / hours / days). So, in the meantime, every code path that will need access to the data may as well look in the local snapshot first.
The question about consistency is an interesting one. The client's view of the authoritative server state may be slightly out of date, when the user issues a request. If certain events happened in the meantime that affect the user's view, then the action can just be kicked back to the user, to be resolved. But 90%+ of the time, the view depends on 10 things that "change rarely", so a cache is a great improvement.
Related issues involve batching / throttling / waiting for already-sent requests to complete.
PS: That was quick. I posted this and literally 10 seconds later it got a downvote.
Re: Downsides of Caching
#7It occurs to me that sharding shares most of these disadvantages. It avoids the problem of "you no longer read from your source of truth", but the overall complexity and set of failure modes looks strikingly similar.
I wonder how many sleepless nights have been caused by combining the two.
Re: Downsides of Caching
#8I agree with pretty much everything in this post, though I would add one more thing. It's not so much a downside of caching as a misuse: Application-level caches should never cache local data. Cache network responses. Cache the results of computations. Don't cache files or disk reads. Operating systems already implement disk caches, and they do a better job of it than you. That's in addition to a modern computer's numerous hardware caches. For example, take this code:
...
FILE *fp = fopen("example.txt", "r");
char dest;
int bytes_read = fread(&dest, 1, 1, fp);
putchar(dest);
...
Re: Downsides of Caching
#9Good article which touches on real issues that a lot of developers won't really appreciate themselves until it happens to them (unless they have a strong background in distributed systems theory, and maybe even not then). A little strange that it doesn't use the word "consistency" even once though. :)
By dropping a cache into an existing system, you're weakening consistency in the name of performance. At best, your strongly-consistent system has started taking on eventually-consistent properties (but maybe not even eventual depending on how you invalidate/expire what's in your cache). Eventual consistency can help you scale, but reasoning about it is really hard.
In some sense caching as described by OP is a tool to implement CAP theorem tradeoffs, and Eric Brewer described the reality of trading off the C (consistency) for A/P (availability/partition-tolerance) better than I ever could:
Another aspect of CAP confusion is the hidden cost of
forfeiting consistency, which is the need to know the
system’s invariants. The subtle beauty of a consistent
system is that the invariants tend to hold even when the
designer does not know what they are. Consequently, a
wide range of reasonable invariants will work just fine.
Conversely, when designers choose A, which requires
restoring invariants after a partition, they must be
explicit about all the invariants, which is both
challenging and prone to error. At the core, this is the
same concurrent updates problem that makes multithreading
harder than sequential programming.Re: Downsides of Caching
#10I agree with all the main points here: caching adds a significantly complex component to the system. You should only do it if you absolutely must pull data closer to a consumer. Adding caching "to pick up quick wins" is always dumb. With that in mind, I do think most of the pitfalls listed here can be avoided with well-understood tools and techniques. There's no real need to be running your cache in-process with your…
There's no real need to be running your cache in-process with your GC'd implementation language.
You can totally argue that engineers should be forced to think about and address these issues up front with more rigor, and in a perfect world I think I'd agree. :)