Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a cache service with a database

avi.im

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Replacing a cache service with a database

#51

Post by chamomeal »

Hey OP you may have seen this already, but in case you didn’t see my other comment, you should definitely check out this talk by Martin Kleppman.

https://youtu.be/fU9hR3kiOK0?si=t9IhfPtCsSyszscf

It details Apache samza, which I didn’t totally grasp but it seems similar to what you’re talking about here.

He talks about how if you could essentially use an event stream as your source of truth instead of a database, and you had a sufficiently powerful stream processor, you could define views on that data by consuming stream events.

The end result is kind of like an auto-updating cache with no invalidation issues or race conditions. Need a new view on the data? Just define it and run the entire event stream through it. Once the stream is processed, that source of data is perpetually accurate and up-to-date.

I’m not a database guy and most of this stuff is over my head, but I loved this talk and I think you should check it out! It’s the first thing I thought of when I read your post.

Re: Replacing a cache service with a database

#52

Post by Alex_L_Wood »

miggy wrote: post #27

We had a critical service that often got overwhelmed, not by one client app but by different apps over time. One week it was app A, the next week app B, each with its own buggy code suddenly spamming the service. The quick fix suggested was caching, since a lot of requests were for the same query. But after debating, we went with rate limiting instead. Our reasoning: caching would just hide the bad behavior and keep…

It's funny how I encountered a problem which went exactly the opposite way! We initially introduced a rate limiter that was adequate for the time, but with the product scaling up it stopped being adequate, and any failures with 429 were either ignored, or closed as client bugs. Only after some time we realized that the rate of requests scaled up approximately with the rate of product growth, and a quick fix was to simply remove the limiter, but after a couple of times when DB decided to take a nap after being overwhelmed, we added a caching layer.

Just goes to show that there is no silver bullet - context, experience and good amount of gut feeling is paramount.

Re: Replacing a cache service with a database

#53

Post by stevoski »

Something missing from the article:

For the type of cache usage described in the article, cache lookups are almost always O(1). This is because a cache value is retrieved for a specific key.

Whereas db queries are often more complicated and therefore take longer. Yes, plenty of db queries are fetching a row by a key, and therefore fast. But many queries use a join and a somewhat complicated WHERE clause.

Re: Replacing a cache service with a database

#54

Post by jiggawatts »

zeras wrote: post #16

I think a fundamental mistake I see many developers make is they use caching trying to solve problems rather than improve efficiency. It's the equivalent of adding more RAM to fix poor memory management or adding more CPUs/servers to compensate for resource heavy and slow requests and complex queries. If your application requires caching to function effectively then you have a core issue that needs to be resolved, an…

Not to mention latency! Caching does nothing to fix the latency of “misses”, which means any app that uses a caching layer to paper over a bad design will forever have a terrible P99 (or even P90) latency.

“But, but, when I reload the page now it’s fast! I fixed it!”

Re: Replacing a cache service with a database

#55

Post by spyspy »

miggy wrote: post #27

We had a critical service that often got overwhelmed, not by one client app but by different apps over time. One week it was app A, the next week app B, each with its own buggy code suddenly spamming the service. The quick fix suggested was caching, since a lot of requests were for the same query. But after debating, we went with rate limiting instead. Our reasoning: caching would just hide the bad behavior and keep…

You can never trust clients to behave. If your goal is to reduce infra cost, sure, rate limiting is an acceptable answer. But is it really that hard to throw on a cache and provision your service to be horizontally scalable?

Re: Replacing a cache service with a database

#56

Post by spyspy »

Alex_L_Wood wrote: post #52
miggy wrote: post #27

We had a critical service that often got overwhelmed, not by one client app but by different apps over time. One week it was app A, the next week app B, each with its own buggy code suddenly spamming the service. The quick fix suggested was caching, since a lot of requests were for the same query. But after debating, we went with rate limiting instead. Our reasoning: caching would just hide the bad behavior and keep…

It's funny how I encountered a problem which went exactly the opposite way! We initially introduced a rate limiter that was adequate for the time, but with the product scaling up it stopped being adequate, and any failures with 429 were either ignored, or closed as client bugs. Only after some time we realized that the rate of requests scaled up approximately with the rate of product growth, and a quick fix was to si…

Something that was drilled into me early in my career was that you cannot expect your cache to be up 100% of the time. The logical extension of that is your main DB needs to be able to handle 100% of your traffic at a moment’s notice. Not only has this kind of thinking saved my ass on several occasions, but it’s also actually kept my code much cleaner. I don’t want to say rate limiters and circuit breakers are the mark of bad engineering, butttt they’re usually just good engineering deferred.

Re: Replacing a cache service with a database

#57

Post by spyspy »

WillDaSilva wrote: post #50
andersmurphy wrote: post #39

Earlier quoted context omitted.

I guess CPUs are pretty buggy with all their caches. If only the hardware people could fix their buggy systems. In all seriousness sometimes a cache is what you need. Inline caching is a classic example.

There are times when a cache is appropriate, but I often find that it's more appropriate for the cache to be on the side of whoever is making all the requests. This isn't applicable when that is e.g. millions of different clients all making their own requests, but rather when we're talking about one internal service putting heavy load on another one. The team with the demanding service can add a cache that's appropri…

You cannot trust your clients. Period. It doesn’t matter if they’re internal or external. If you design (and test!) with this assumption in mind, you’ll never have a bad day. I’ve really never understood why teams and companies have taken this defensive stance that their service is being “abused” despite having nothing even resembling an SLA. It seemed pretty inexcusable to not have a horizontally scaling service back in 2010 when I first started interning at tech companies, and I’m really confused why this is still an issue today.

Re: Replacing a cache service with a database

#58

Post by WillDaSilva »

spyspy wrote: post #57
WillDaSilva wrote: post #50

Earlier quoted context omitted.

There are times when a cache is appropriate, but I often find that it's more appropriate for the cache to be on the side of whoever is making all the requests. This isn't applicable when that is e.g. millions of different clients all making their own requests, but rather when we're talking about one internal service putting heavy load on another one. The team with the demanding service can add a cache that's appropri…

You cannot trust your clients. Period. It doesn’t matter if they’re internal or external. If you design (and test!) with this assumption in mind, you’ll never have a bad day. I’ve really never understood why teams and companies have taken this defensive stance that their service is being “abused” despite having nothing even resembling an SLA. It seemed pretty inexcusable to not have a horizontally scaling service bac…

I fully agree. The rate limits are how you control the behaviour of the clients. My suggestion of leaving caching to the clients, which they may want to do in order to avoid hitting the rate limit.

Re: Replacing a cache service with a database

#59

Post by ajcp »

chamomeal wrote: post #51

Hey OP you may have seen this already, but in case you didn’t see my other comment, you should definitely check out this talk by Martin Kleppman. https://youtu.be/fU9hR3kiOK0?si=t9IhfPtCsSyszscf It details Apache samza, which I didn’t totally grasp but it seems similar to what you’re talking about here. He talks about how if you could essentially use an event stream as your source of truth instead of a database, and…

Thank you for sharing. I thoroughly enjoyed the talk and am as well not a "database guy".

Re: Replacing a cache service with a database

#60

Post by shivasaxena »

chamomeal wrote: post #51

Hey OP you may have seen this already, but in case you didn’t see my other comment, you should definitely check out this talk by Martin Kleppman. https://youtu.be/fU9hR3kiOK0?si=t9IhfPtCsSyszscf It details Apache samza, which I didn’t totally grasp but it seems similar to what you’re talking about here. He talks about how if you could essentially use an event stream as your source of truth instead of a database, and…

[dead]
Post reply on HN