Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a cache service with a database

avi.im

Post reply on HN
Print view View original

41–50 of 69 posts

Re: Replacing a cache service with a database

#41

Post by jedberg »

barrkel wrote: post #14
hoppp wrote: post #3

The cache service is a database of sorts that usually stores key value pairs. The difference is in persistence and scaling and read/write permissions

No, what makes a cache a cache is invalidation. A cache is stale data. It's a latent out of date calculation. It's misinformation that risks surviving until it lies to the user.

This is true but a lot of the trouble in invalidation can be avoided by using smarter cache keys.

For example, on reddit, fully rendered comments are cached, so that the renderer doesn't have to redo its work. But the cache key includes the date of the last edit on the comment, which is already known when requesting the value from the cache. In this way, you never have to invalidate that key, because editing the comment makes a new key. The old one will just get ejected eventually.

Re: Replacing a cache service with a database

#42

Post by IgorPartola »

hinkley wrote: post #33
IgorPartola wrote: post #15

Earlier quoted context omitted.

If you think of it as a cache, yes. If you think of it as another data layer then no. For example, let’s say that every web page your CMS produces is created using a computationally expensive compilation. But the final product is more or less static and only gets updated every so often. You can basically have your compilation process pull the data from your source of truth such as your RSBMS but then store the final…

No. It’s not a data layer, it’s global shared state. Global shared state always has consequences. Sometimes the consequences are worth the trouble. But it is trouble. If you think about Source of Truth, System of Record, cache is neither of those, and sits between them. There’s a lot of problems you can fix instead by improving the SoT or SoR situation in that area if the code.

Hard disagree. Having used the architecture I described in large practical deployments it works way better than what you are making it out to be. But I don’t know the domain you work in and your constraints so it is possible that for you it would not work.

Re: Replacing a cache service with a database

#43

Post by IgorPartola »

hinkley wrote: post #35
lemmsjid wrote: post #25

Earlier quoted context omitted.

Quite agree, this is how I explain it to people. When you think of cache as another derived dataset then you start to realize that the issues caches bring to architectures are often the result of not having an agreement between the business and engineering on acceptable data consistency tolerances. For example, outside the world of caching, if you email users a report, and the data is embedded in the email, then you…

The cache is an incomplete dataset by definition. It’s not a data set, it’s a cache of a data set. You can never ensure you get a clean read of the system state from the cache because it’s never in sync and has gaps.

What about materialized views? CPU cache? Only the Sith deal in absolutes :)

Re: Replacing a cache service with a database

#44

Post by IgorPartola »

cpursley wrote: post #34
IgorPartola wrote: post #26

Earlier quoted context omitted.

Sorry you lack the imagination to substitute your preferred data store into what I wrote. Hope it gets easier.

I'll never have enough imagination to believe mongo is a good solution. Postgres has jsonb, vector type; redis is a fine-enough cache. Why use a known junk "database" when there are superior solutions and truly open source?

I didn’t say you have to use it. I said you could. Or any other data store that fits your use case. I used a MongoDB instance back in 2012 in a serious production environment in this exact way and it worked flawlessly while Postgres was what gave us trouble (it had a bunch of features added since that would have made those issues disappear but back then it didn’t have built in replication for example.)

But again this is not an endorsement of MongoDB. I wouldn’t use it today but I did use it successfully and that company and tech stack sold for quite a bit of money and the software still runs, though I’m not sure on what stack. Again, if you are stuck on this one part of my comment… can’t help you.

Re: Replacing a cache service with a database

#46

Post by tootie »

simonw wrote: post #6

A friend of mine once argued that adding a cache to a system is almost always an indication that you have an architectural problem further down the stack, and you should try to address that instead. The more software development experience I gain the more I agree with him on that!

Most of the time I use caching it's to cut down on network round trips. If I'm fetching data on every end user request that only updates daily or weekly caching that's a no-brainer. Edge caching for content sites is also a no-brainer. Caching something computationally expensive may be fishy but also may be useful. Even if you are just papering over some inefficient process, that's not necessarily a sin. Sometimes you have to be pragmatic.

Re: Replacing a cache service with a database

#47

Post by zeras »

chamomeal wrote: post #20
zeras wrote: post #16

I think a fundamental mistake I see many developers make is they use caching trying to solve problems rather than improve efficiency. It's the equivalent of adding more RAM to fix poor memory management or adding more CPUs/servers to compensate for resource heavy and slow requests and complex queries. If your application requires caching to function effectively then you have a core issue that needs to be resolved, an…

Idk I think caching is a crucial part of many well-designed systems. There’s a lot of very cache-able data out there. If invalidating events are well defined or the data is fine being stale (week/month level dashboards, for example), that’s a fantastic reason to use a cache. I’d much rather just stuff those values in a cache than figure out any other more complicated solution. I also just think it’s a necessary evil…

Caching is definitely a useful and even a key component to producing efficent and high performance applications and services.

I think the mistake is not using caching, but rather using it too soon in the development process.

There are times when caching is a requirement because there is simply no way to provide efficient performance without it, but I think too many times developers jump straight to caching without thinking because it solves potential problems for them before they happen.

The real problem comes later though at scale when caching can no long compensate for the development inefficiencies.

Now the developers have to start rewriting core code which will take time to thoroughly complete and test and/or the engineers have to figure out a way to throw more resources at the problem.

Re: Replacing a cache service with a database

#48

Post by hinkley »

IgorPartola wrote: post #43
hinkley wrote: post #35

Earlier quoted context omitted.

The cache is an incomplete dataset by definition. It’s not a data set, it’s a cache of a data set. You can never ensure you get a clean read of the system state from the cache because it’s never in sync and has gaps.

What about materialized views? CPU cache? Only the Sith deal in absolutes :)

CPU cache means that the same value read twice will return the same value. Some exceptions for NUMA, and mu[tiple threads. But two reads of a cache cache make no such guarantees.

There is a vast number of undiagnosed race conditions in modern code cause by cache eviction in the middle of 'transactions' under high system load.

Re: Replacing a cache service with a database

#50

Post by WillDaSilva »

andersmurphy wrote: post #39
miggy wrote: post #27

We had a critical service that often got overwhelmed, not by one client app but by different apps over time. One week it was app A, the next week app B, each with its own buggy code suddenly spamming the service. The quick fix suggested was caching, since a lot of requests were for the same query. But after debating, we went with rate limiting instead. Our reasoning: caching would just hide the bad behavior and keep…

I guess CPUs are pretty buggy with all their caches. If only the hardware people could fix their buggy systems. In all seriousness sometimes a cache is what you need. Inline caching is a classic example.

There are times when a cache is appropriate, but I often find that it's more appropriate for the cache to be on the side of whoever is making all the requests. This isn't applicable when that is e.g. millions of different clients all making their own requests, but rather when we're talking about one internal service putting heavy load on another one.

The team with the demanding service can add a cache that's appropriate for their needs, and will be motivated to do so in order to avoid hitting the rate limit (or reduce costs, which should be attributed to them).

Post reply on HN