41–46 of 46 posts
Can someone please explain, without technical buzzwords, what Materialize is? I checked the docs but they mention streaming DB, Timely and Differential Dataflow which I don't know either.
It lets you create materialized views that are automatically updated in an incremental way as underlying data changes, without recomputing the whole query. Previously they supported running those queries on Postgres/Kafka, now they've added their own persistence layer and horizontal scalability.
Earlier quoted context omitted.
Such a shame, I had an ideal use case for a product I'm working on, but there's no way I'm shackling it to cloud vendors. Just not a good fit for this product. I really hope materialize eventually inspires something like it in the open source world.
Please also take a look at https://github.com/risingwavelabs/risingwave if you are looking for advanced streaming databases. It is under Apache License and also support on-prem deployment (docker, kubernetes) with full function set of distributed clustering, compute-storage disaggregation, etc.. Let me know if anything.
Earlier quoted context omitted.
I have worked at Materialize since 2019. The elevator pitch is that it is a database that lets you maintain the results of queries in memory (and now S3) and have them update in real time so the current result is always available.
Can you describe some use cases? The only one that comes to mind is powering dashboards. That said, do you support persistent tables or time travel? (E.g., accessing the contents of the view as it was at time t). If not, how should a situation where multiple independent clients read the view be handled? If two people load the same dashboard but see differences based on when the table was read, that might cause confus…
Yes
> or time travel
No, but I think the changes that need to be made in order to do so are tractable, so I expect we will eventually (I’m not an expert in this area of the code, so please don’t take this as an official statement).
I use PG with an alternative materialized views implementation[0] that is pure PlPgSQL and that exposes real tables that can be used to write to in triggers, and where the views can be marked stale too. This means hand-coding triggers to keep the materializations up to date, or else to mark them as out of date (because maybe some operations would be slow or hard to hand-code triggers for), but this works remarkably w…
I use PG with an alternative materialized views implementation[0] that is pure PlPgSQL and that exposes real tables that can be used to write to in triggers, and where the views can be marked stale too. This means hand-coding triggers to keep the materializations up to date, or else to mark them as out of date (because maybe some operations would be slow or hard to hand-code triggers for), but this works remarkably w…
You might be interested in https://github.com/sraoss/pg_ivm
The poor man's version of Materialize that I implemented is the following: Step 1) Find all "paths" between tables * Use the postgres information schema to get all relations * Use npm library graph-cycles to see if there are any graph cycles. If so.. some relations go on a blacklist. * Use npm library topopsort to sort the graph * Traverse the graph and find all possible paths from and to tables * Generate SQL querie…