Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a cache service with a database

avi.im

Post reply on HN
Print view View original

61–69 of 69 posts

Re: Replacing a cache service with a database

#61

Post by miggy »

spyspy wrote: post #55
miggy wrote: post #27

We had a critical service that often got overwhelmed, not by one client app but by different apps over time. One week it was app A, the next week app B, each with its own buggy code suddenly spamming the service. The quick fix suggested was caching, since a lot of requests were for the same query. But after debating, we went with rate limiting instead. Our reasoning: caching would just hide the bad behavior and keep…

You can never trust clients to behave. If your goal is to reduce infra cost, sure, rate limiting is an acceptable answer. But is it really that hard to throw on a cache and provision your service to be horizontally scalable?

Scaling matters, but why pay for abusive clients or bots? Adding a cache is easy; the hard part is invalidation, sync, and thundering herd. Use it if the product needs it, not as a band-aid.

Re: Replacing a cache service with a database

#62

Post by mannyv »

zeras wrote: post #16

I think a fundamental mistake I see many developers make is they use caching trying to solve problems rather than improve efficiency. It's the equivalent of adding more RAM to fix poor memory management or adding more CPUs/servers to compensate for resource heavy and slow requests and complex queries. If your application requires caching to function effectively then you have a core issue that needs to be resolved, an…

If your database is slow because it's on spinning disks, then a cache will speed up access.

That's not a fundamental mistake, and there's very little you can do about that from an efficiency point of view.

It's easy to forget that there was a world without SSDs, high speed pipes, etc - but it actually did exist. And that wasn't so long ago either.

And of course sometimes putting data nearer to the user actually makes sense...like the Netflix movie boxes inside various POPs or CDNs. Bandwidth and latency are actual factors for many applications.

That said, most applications probably should investigate adding indexes to their databases (or noSQL databases) instead of adding a cache layer.

Re: Replacing a cache service with a database

#63

Post by pixl97 »

spyspy wrote: post #57
WillDaSilva wrote: post #50

Earlier quoted context omitted.

There are times when a cache is appropriate, but I often find that it's more appropriate for the cache to be on the side of whoever is making all the requests. This isn't applicable when that is e.g. millions of different clients all making their own requests, but rather when we're talking about one internal service putting heavy load on another one. The team with the demanding service can add a cache that's appropri…

You cannot trust your clients. Period. It doesn’t matter if they’re internal or external. If you design (and test!) with this assumption in mind, you’ll never have a bad day. I’ve really never understood why teams and companies have taken this defensive stance that their service is being “abused” despite having nothing even resembling an SLA. It seemed pretty inexcusable to not have a horizontally scaling service bac…

>why teams and companies have taken this defensive stance that their service is being “abused” despite having nothing even resembling an SLA.

I mean because bad code on a fast client system can cause a load higher than all other users put together. This is why half the internet is behind something like cloudflare these days. Limiting, blocking, and banning has to be baked in.

Re: Replacing a cache service with a database

#64

Post by sellmesoap »

spyspy wrote: post #56
Alex_L_Wood wrote: post #52

Earlier quoted context omitted.

It's funny how I encountered a problem which went exactly the opposite way! We initially introduced a rate limiter that was adequate for the time, but with the product scaling up it stopped being adequate, and any failures with 429 were either ignored, or closed as client bugs. Only after some time we realized that the rate of requests scaled up approximately with the rate of product growth, and a quick fix was to si…

Something that was drilled into me early in my career was that you cannot expect your cache to be up 100% of the time. The logical extension of that is your main DB needs to be able to handle 100% of your traffic at a moment’s notice. Not only has this kind of thinking saved my ass on several occasions, but it’s also actually kept my code much cleaner. I don’t want to say rate limiters and circuit breakers are the ma…

Reminds me of gas plumbing, the indoor lines are only a few psi above ambient, but the lines themselves have to take line pressure to 300psi is case the regulator fails. It's good advice!

Re: Replacing a cache service with a database

#65

Post by interstice »

I've been thinking a lot recently about edge/client layer data sync, interesting to hear where others are at. Noria seems to have got as far as a smart way to store and manage tabular data, however this doesn't seem to help much when the frontend is built on blobs & if one isn't prepared to write the additional layer for read/write on top of the rest of the fetching system.

The dumb/MVP approach I'd like to try sometime is close-to-client read only sqlite db's that get managed in the background and neatly handled by wrapper functions around things like fetch. The part I've been slowly thinking about is Noria style efficient handling of data structures while allowing for 'raw' queries, ideally I'd like to set this up so the frontend doesn't need an additional layers worth of read/write functionality just to have CDN-like behaviour. Maybe something like plugins to [de/re]normalise different kinds of blob to tables (from gql, groqd, etc). I'd also like to include a realtime cache invalidation/update system to keep all clients in sync without cache clearing... If I ever get that far.

Re: Replacing a cache service with a database

#66

Post by Supermancho »

hinkley wrote: post #36
Supermancho wrote: post #12

Earlier quoted context omitted.

ie A cache is a database. The difference is features and usage.

A database is usually a union of all of the questions that can be asked about a topic. A cache by definition is a subset of that. Subsets are not the sets. And if you treat them as if they are, which 90% of people do, you’re gonna have a bad time.

> A database is usually a union of all of the questions that can be asked about a topic

That's some AI level sophism.

A database is a durable store of data that can be modified and read. Ostensibly, we're talking about computer databases. You can define the soft terms at your leisure and to suit your needs. There are many categories of discussion that will never intersect with this definition. Communication is not a database. Art is not a database. History is not a database. Medicine is not a database. et al.

A cache is a database. Differentiating a cache and database by label is a misnomer.

Re: Replacing a cache service with a database

#67

Post by interstice »

interstice wrote: post #65

I've been thinking a lot recently about edge/client layer data sync, interesting to hear where others are at. Noria seems to have got as far as a smart way to store and manage tabular data, however this doesn't seem to help much when the frontend is built on blobs & if one isn't prepared to write the additional layer for read/write on top of the rest of the fetching system. The dumb/MVP approach I'd like to try somet…

This got me thinking a bit more. Rest / GraphQL / Groq handled with adapters, flatten anything nested that references an ID to the row level. Opinionated queries (queries only fetch a superset/subset of the same structure). Fetched data 'fans out' the new content into the rows based on ID to fill out/update structure. Lives in a service worker or side by side with frontend. Drops oldest/least fetched data when limits are reached. Would something like that work?

Alternatively just ship an entire shallow copy of least changed / most used data as sqlite db's to the edge, push updates to those, and fetch from source anything that isn't in the DB. Might be simpler.

Re: Replacing a cache service with a database

#68

Post by hinkley »

Supermancho wrote: post #66
hinkley wrote: post #36

Earlier quoted context omitted.

A database is usually a union of all of the questions that can be asked about a topic. A cache by definition is a subset of that. Subsets are not the sets. And if you treat them as if they are, which 90% of people do, you’re gonna have a bad time.

> A database is usually a union of all of the questions that can be asked about a topic That's some AI level sophism. A database is a durable store of data that can be modified and read. Ostensibly, we're talking about computer databases. You can define the soft terms at your leisure and to suit your needs. There are many categories of discussion that will never intersect with this definition. Communication is not a…

> That's some AI level sophism.

Oh fuck off. Calling everything AI is so 2024.

A database is a system of record. It can also be a source of truth. A cache is neither. Treating it as one is dangerous. Insisting others should is idiocy.

Re: Replacing a cache service with a database

#69

Post by Supermancho »

hinkley wrote: post #68
Supermancho wrote: post #66

Earlier quoted context omitted.

> A database is usually a union of all of the questions that can be asked about a topic That's some AI level sophism. A database is a durable store of data that can be modified and read. Ostensibly, we're talking about computer databases. You can define the soft terms at your leisure and to suit your needs. There are many categories of discussion that will never intersect with this definition. Communication is not a…

> That's some AI level sophism. Oh fuck off. Calling everything AI is so 2024. A database is a system of record. It can also be a source of truth. A cache is neither. Treating it as one is dangerous. Insisting others should is idiocy.

> A database is a system of record. It can also be a source of truth.

This is meaningless. A cache is used in lieu of the value because it's considered equivalent.

> Insisting others should is idiocy.

I did no such thing. Good luck with whatever.

Post reply on HN