Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Readyset: A MySQL and Postgres wire-compatible caching layer

github.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#41

Post by marceloaltmann »

KingOfCoders wrote: post #40

This might be good tech and a good company. Once we used a distributed caching system in a startup which was open source. Then the open source version got cut features we needed, so we bought a license. Then the startup was bought up by a large software company and the license costs went 10x YoY with a one week notice. As our migration away from this tech was not done, because it was very complicated and tied into ou…

That is a good point on the application changes. What is appealing from Readyset is that it does not require you to change your application code. You can just change your database connection string to point to it and it will start to proxy your queries to your database. From there you can choose what you want to cache, and everything else (writes, non supported queries, non cached read queries) will be automatically proxied to your database.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#42

Post by gvsg-rs »

PeterZaitsev wrote: post #37
gvsg-rs wrote: post #33

Earlier quoted context omitted.

[dead]

BSL is not Open Source license. The fact outdated code Eventually becomes Open Source does not make it Open Source license It is great ReadySet is Source available and free to use in number of circumstances but I wish you would avoid misleading marketing it as Open Source

I apologize. I unintentionally misspoke. You are correct that BSL is a Source Available license as opposed to an Open Source license.

I don't believe the website or the Github page makes that claim.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#43

Post by hobofan »

zer00eyz wrote: post #27
hobofan wrote: post #8

Earlier quoted context omitted.

You are reading from "fresh" (though only eventually consistent) data. For many public facing queries that's enough. After an initial query seen by their proxy, you can configure all future queries of the same kind to be pre-computed. So I think it makes more sense to think of it like an auto-updating materialized view available with the click of a button rather than a cache. There are some more under-the hood detail…

Normal request: Request -> work -> work.. -> query/db Efective Cache: Request -> (less)work -> cache This Product: Request -> work -> work.. -> query/cache I understand the concept of caching at a boundary layer. I fail to see the point of cache at THIS boundary layer. You have all the problems of a cache with fewer benefits (you're not going to fix a thundering Hurd at this level).

> You have all the problems of a cache with fewer benefits

No you don't. You have next to none of the problems of a cache (especially as you directly have to opt-in individual queries to it) like cache invalidation, etc., with all the benefits. It's about as free as performance benefits can be (from a implementations standpoint).

I understand that for many use-cases caching at the response level may be preferable, but there are also many use-cases that are read-heavy but also involve a lot of computed values that are updated and have to be recomputed regularly, where this dataflow-based approach has been shown to be one of the least compute intensive and efficient solutions.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#44

Post by thom »

Someone knowledgeable might know: is this just incremental view updates? To what extent is the cache intelligent if parameters, where clauses, or aggregates change?

I really love this space and have been impressed with Materialize, but even if you can make some intermediate state incremental, if your workload is largely dynamic you end up needing to jump the whole way to OLAP platforms. I’m hopeful that we’re closer and closer to having our cake and eating it here, and that the operational data warehouse is only round the corner.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#45

Post by dspillett »

gilbertbw wrote: post #36
BrentOzar wrote: post #34

In the Microsoft SQL Server space, several of these vendors have come and gone. My clients have been burned badly by 'em, so a few quick lessons learned: Be aware that there are hundreds of open issues[0] and dozens of pull requests [1], some of which involve clients being unable to connect or not supporting all components of the SQL language. Just because your database supports something, doesn't mean your caching l…

Regarding the new database version issue, I wonder why the caching layer can't just pass any query it is unable to process on to the underlying database? This would be more complex if the feature you are using does not return a normal table of results back (e.g. the pub/sub support in Postgres).

> I wonder why the caching layer can't just pass any query it is unable to process on to the underlying database?

A caching layer can do that, and I'd be surprised if serious ones don't. It might not be perfect when moving between DB versions though: the new version might perhaps introduce syntax that is mistaken for something else which the layer _thinks_ it understands. While this can be tested for to a large extent by the devs having access to alph/beta/rc versions, but I can still see things slipping through and you are relying on people keeping their versions of the caching layer as up-to-date as everything else which is not a given.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#47

Post by hobofan »

BrentOzar wrote: post #34

In the Microsoft SQL Server space, several of these vendors have come and gone. My clients have been burned badly by 'em, so a few quick lessons learned: Be aware that there are hundreds of open issues[0] and dozens of pull requests [1], some of which involve clients being unable to connect or not supporting all components of the SQL language. Just because your database supports something, doesn't mean your caching l…

I don't think it's fair to hold the number of open issues and pull requests against them. Looking through them for a minute, it looks like 95%+ are from their own team members, with a good chunk of the issues being "low priority" issues. So you are just seeing the typical ever-growing backlog that is normally in a private JIRA instance.

Having said that, the way they work with pull requests is unlike anything else I've seen. I see that they are using a merge bot, but apart from that all branch names are completely illegible. As a lot of the team seems to be present in this thread, it would be interesting to get some details about that.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#48

Post by BrentOzar »

gilbertbw wrote: post #36
BrentOzar wrote: post #34

In the Microsoft SQL Server space, several of these vendors have come and gone. My clients have been burned badly by 'em, so a few quick lessons learned: Be aware that there are hundreds of open issues[0] and dozens of pull requests [1], some of which involve clients being unable to connect or not supporting all components of the SQL language. Just because your database supports something, doesn't mean your caching l…

Regarding the new database version issue, I wonder why the caching layer can't just pass any query it is unable to process on to the underlying database? This would be more complex if the feature you are using does not return a normal table of results back (e.g. the pub/sub support in Postgres).

> I wonder why the caching layer can't just pass any query it is unable to process on to the underlying database?

Because sometimes it's not about the query you pass in - it's about a new data type, for example. The layer may think it understands the query, but may not be able to handle the format of the results, or may cache them incorrectly.

For example, when SQL Server added the JSON data type, one of my customers found that the caching layer mangled it for some clients, but not others, or would return the correct result the first time, but the cached result was formatted incorrectly.

Another problem can be the connection itself, like when SQL Server added support for Always Encrypted, an end-to-end encryption method.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#49

Post by alwaysrusty »

From a tech perspective, this is really cool. From a use case perspective, could someone help me understand why a developer would adopt something like this over a database like Clickhouse, outside of some fintech use cases where milliseconds of latency really matter? I'd be worried about introducing an additional point of failure to the data stack. And, if this is like Materialize, I'd be worried about this not suppporting ad hoc queries -- only precomputed ones.
Post reply on HN