Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Readyset: A MySQL and Postgres wire-compatible caching layer

github.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#21

Post by exabrial »

One of the things we love about using JPA (We use EclipseLink) is it comes with caching, for free, and it’s transparent. You can mark any field as a cache index and it automatically tries the cache first. Updates are published and loaded into every nodes cache automatically, and you get fallback protection in the form of incremental version numbers on rows.

The one thing it can’t handle however is range update queries or native queries that perform updates.

You can just avoid them in your architecture… OR maybe this is the solution we’ve been looking we’ve been looking for! definitely going to give this a spin!

Documentation looks very complete and I like there’s a UI to view the query cache.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#22

Post by lukoktonos »

jitl wrote: post #18
lukoktonos wrote: post #16

Earlier quoted context omitted.

What do you mean by write support here? Readyset will apply writes via a replication stream from the upstream database.

I think they mean acting as a write-through cache. If you send writes to the cache and have the cache delegate those writes to the underlying DB you get consistent invalidation, and if the write-through cache is smart enough it can even be transactional with the underlying source of truth.

Readyset does have some work and design done towards read-your-writes consistency, but it's still under development. The Noria paper does briefly touch on extending partial materialization with stronger consistency models via MVCC or other mechanisms as well.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#23

Post by gwbas1c »

zer00eyz wrote: post #4

I dont understand what the use case is for this. If I have a front end, I would hope that the formated response is what were caching. Be that HTML or JSON. If I cant read from that cache then I should be reading from fresh data all together? right?

Ever add (or inherit) the server-side part of an application that uses Redis or Memcache? The data is denormalized: When you do an update/insert/delete into the SQL side of things, you need to do a corresponding change in Redis / Memcache. All of your queries end up being something like: Try Redis/Memcache, if the data isn't present, query the database and insert into Redis/Memcache.

It (Redis/Memcache caching) adds a huge amount of complexity to your application, and the risk of defects is very, very real.

ReadySet basically gives you a magic "stick this thing between your database and application and we'll do the caching for you." It totally eliminates a time consuming and error-prone part of your application.

If you want to, they go into details here: https://blog.readyset.io/dont-use-kv-stores/

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#27

Post by zer00eyz »

hobofan wrote: post #8
zer00eyz wrote: post #4

I dont understand what the use case is for this. If I have a front end, I would hope that the formated response is what were caching. Be that HTML or JSON. If I cant read from that cache then I should be reading from fresh data all together? right?

You are reading from "fresh" (though only eventually consistent) data. For many public facing queries that's enough. After an initial query seen by their proxy, you can configure all future queries of the same kind to be pre-computed. So I think it makes more sense to think of it like an auto-updating materialized view available with the click of a button rather than a cache. There are some more under-the hood detail…

Normal request: Request -> work -> work.. -> query/db

Efective Cache: Request -> (less)work -> cache

This Product: Request -> work -> work.. -> query/cache

I understand the concept of caching at a boundary layer. I fail to see the point of cache at THIS boundary layer. You have all the problems of a cache with fewer benefits (you're not going to fix a thundering Hurd at this level).

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#29

Post by dpcx »

Pretty sure that this is the database that Jon Gjengset[0] was working on as part of his thesis project. There have been several videos shared by him during talks about the system. It's a really interesting concept.

edit: Here's[1] a video where he talks about the concept

[0]: https://www.youtube.com/@jonhoo [1]: https://www.youtube.com/watch?v=GctxvSPIfr8

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#30

Post by jitl »

10000truths wrote: post #24

This sounds like it has heavy overlap with IVM. How does Readyset distinguish itself from existing solutions like pg_ivm or Materialize?

ReadySet descends from Noira (https://www.usenix.org/conference/osdi18/presentation/gjengs...), which I view as the next generation of incremental dataflow technology after Naiad/Differential Dataflow/Materialize.

The key difference is that Noira/ReadySet supports partial materialization and can reconstruct data on-demand, whereas Naiad/Differential Dataflow/Materialize must keep a complete materialization up to date. ReadySet can partially evict parts of the data flow and bring them back later if needed.

In practical terms if you have a materialized view you want to maintain, in ReadySet you pay O(part of the materialized data flow you actually need), which is less than O(entire materialized view) you’d pay with Materialize.

Post reply on HN