Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Readyset: A MySQL and Postgres wire-compatible caching layer

github.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#61

Post by Jonhoo »

hobofan wrote: post #53
thom wrote: post #44

Someone knowledgeable might know: is this just incremental view updates? To what extent is the cache intelligent if parameters, where clauses, or aggregates change? I really love this space and have been impressed with Materialize, but even if you can make some intermediate state incremental, if your workload is largely dynamic you end up needing to jump the whole way to OLAP platforms. I’m hopeful that we’re closer…

They have a bit about their technical foundation here[0]. Given that Readyset was co-founded by Jon Gjengset (but has apparently since departed the company), who authored the paper on Noria[1], I would assume that Readyset is the continuation of that research. I wouldn't call that "just" incremental view maintenance, as it's a lot more involved than the simplest implementation of IVM (though obviously that is the end…

Just poking my head in to say that I technically never departed ReadySet — what happened was that I co-founded the company, but was so burnt-out when it came to databases after my PhD that I decided to leave the running of the company to others. Then, US visa regulations made it so that I couldn't really be involved _at all_ if I wasn't an actual employee, which meant I truly was "just a founder" with no real involvement in the company's execution if you will. Now that I'm back in Europe, that's changing a bit, and I have regular calls with the CEO and such!

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#62

Post by Jonhoo »

AYBABTME wrote: post #56

Love the work. Looks quite similar to what PlanetScale Boost does[1]. Basically the same but as a front-end to someone's existing database? (disclaimer: I work at PS). [1]: https://planetscale.com/blog/how-planetscale-boost-serves-yo...

They're both based on the techniques outlined in the Noria paper (https://www.usenix.org/conference/osdi18/presentation/gjengs...) and my thesis (https://jon.thesquareplanet.com/papers/phd-thesis.pdf), so not terribly surprising they carry some resemblance :p

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#63

Post by AYBABTME »

Jonhoo wrote: post #61
hobofan wrote: post #53

Earlier quoted context omitted.

They have a bit about their technical foundation here[0]. Given that Readyset was co-founded by Jon Gjengset (but has apparently since departed the company), who authored the paper on Noria[1], I would assume that Readyset is the continuation of that research. I wouldn't call that "just" incremental view maintenance, as it's a lot more involved than the simplest implementation of IVM (though obviously that is the end…

Just poking my head in to say that I technically never departed ReadySet — what happened was that I co-founded the company, but was so burnt-out when it came to databases after my PhD that I decided to leave the running of the company to others. Then, US visa regulations made it so that I couldn't really be involved _at all_ if I wasn't an actual employee, which meant I truly was "just a founder" with no real involve…

Thanks so much for your work! Visa problems are a bane on the US. I hope that this situation will improve for you and get better in the future.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#64

Post by rbranson »

marceloaltmann wrote: post #35
gvsg-rs wrote: post #33

Earlier quoted context omitted.

[dead]

On top of that, quoting @martypitt reply: > Most commonly the restrictions prevent you from launching a competing offering. In their case, you can't offer database-as-a-service using their code. Meaning the self hosted version is free to use in any number of servers having in mind the competing offering restriction.

Many people's biggest issue with BSL is that there are dozens of versions of the "additional use grant" which each have bespoke language with very critical clauses, none of which have any case law behind them (correct me if I'm wrong).

Even though software may be licensed under "BSL" it isn't really a standard, even though proponents tend to use the term "BSL" as if it is the same as talking about "GPL" or "BSD". The MariaDB BSL used here is quite different than the HashiCorp BSL, for instance.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#65

Post by ltbarcly3 »

Trying to add transparent caching to a transactional database is just a bad idea and cannot work. Anyone who says it works for them is just in the period after putting it in place and before when they realize why it cannot work.

If it was possible to just slap a cache in between you and the db and magically make shit fast, DB vendors would have done that 20 years ago. Billions of dollars a year is put into relational db development. Papers are published every week, from theoretical ways to model and interact with data to practical things like optimizing query execution plans.

Unless Readyset can point to a patent or a paper that has fundamentally revolutionized how database will be built from today forward it is going to be crap and will burn you.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#66

Post by darkwater »

larsnystrom wrote: post #13

They have a pretty good writeup on why you’d want to use this here: https://blog.readyset.io/dont-use-kv-stores/

To be honest, I see more like a write-up about the shortcomings of using a classic cache system in front of your DB rather than about what Ready set does. Yes, it explains it but it could be better (for example, practical example of how you set a query to be cached, an overview how replication stream ingestion works under the hood etc)

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#67

Post by SahAssar »

marceloaltmann wrote: post #35
gvsg-rs wrote: post #33

Earlier quoted context omitted.

[dead]

On top of that, quoting @martypitt reply: > Most commonly the restrictions prevent you from launching a competing offering. In their case, you can't offer database-as-a-service using their code. Meaning the self hosted version is free to use in any number of servers having in mind the competing offering restriction.

In this case the additional use grant is:

  Additional Use Grant: You may make use of the Licensed Work, provided that
  you may not use the Licensed Work for a Database Service. A ‘Database
  Service’ is a commercial offering that allows third parties (other than
  your employees and contractors) to access the functionality of the Licensed Work.
IANAL but to me that sounds like it could be interpreted as almost any commercial work based on it since some part will "access the functionality", indirectly or directly. I know that is not what they intended, but the language is loose enough to allow almost any interpretation.

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#68

Post by jauntywundrkind »

hobofan wrote: post #53
thom wrote: post #44

Someone knowledgeable might know: is this just incremental view updates? To what extent is the cache intelligent if parameters, where clauses, or aggregates change? I really love this space and have been impressed with Materialize, but even if you can make some intermediate state incremental, if your workload is largely dynamic you end up needing to jump the whole way to OLAP platforms. I’m hopeful that we’re closer…

They have a bit about their technical foundation here[0]. Given that Readyset was co-founded by Jon Gjengset (but has apparently since departed the company), who authored the paper on Noria[1], I would assume that Readyset is the continuation of that research. I wouldn't call that "just" incremental view maintenance, as it's a lot more involved than the simplest implementation of IVM (though obviously that is the end…

Wow, uh, Noria got listed a bunch of times looking back in search. https://hn.algolia.com/?q=noria

(It turns into Nokia results part way through page 2, but still a lot of submissions!)

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#69

Post by gwbas1c »

BrentOzar wrote: post #34

In the Microsoft SQL Server space, several of these vendors have come and gone. My clients have been burned badly by 'em, so a few quick lessons learned: Be aware that there are hundreds of open issues[0] and dozens of pull requests [1], some of which involve clients being unable to connect or not supporting all components of the SQL language. Just because your database supports something, doesn't mean your caching l…

They use the database's replication API. In theory, it should avoid situations where the cache doesn't understand query syntax.

But, what I'd worry about are situations where an application needs to cache an object that's expensive to build. (And perhaps expensive to build because the programmer doesn't understand how to optimize SQL.)

Re: Readyset: A MySQL and Postgres wire-compatible caching layer

#70

Post by jitl »

benesch wrote: post #59
jitl wrote: post #30

Earlier quoted context omitted.

ReadySet descends from Noira ( https://www.usenix.org/conference/osdi18/presentation/gjengs... ), which I view as the next generation of incremental dataflow technology after Naiad/Differential Dataflow/Materialize. The key difference is that Noira/ReadySet supports partial materialization and can reconstruct data on-demand, whereas Naiad/Differential Dataflow/Materialize must keep a complete materialization up to da…

(Materialize CTO here.) Partial materialization is indeed Noria's major contribution to dataflow technology, and it's impressive stuff. But I want to call out that there are a number of techniques that folks use with Materialize to avoid paying for O(entire materialized view). The two most common techniques are demand-driven queries using lateral joins [0] and temporal filters [1]. Noria's approach to partial materia…

Thanks for the great reply, I didn’t know about lateral join.
Post reply on HN