Live data from Hacker News

Materialize: A Streaming Data Warehouse

materialize.io

101–103 of 103 posts

Re: Materialize: A Streaming Data Warehouse

#101

Earlier quoted context omitted.

> Debezium writes them to a side-topic, which isn't obviously synchronized with the data topics That's unfortunate. It seems like it would be ideal to feed schema changes through the same topic, so that the ordering is defined without the usual distributed-system hilarity.

Agreed! We plan to contribute to the Debezium project over the next months along these lines. We’ll have more to say on this shortly!

Nice, looking forward to your contributions. Let me know if you'd like to discuss anything upfront. On a general note, have you considered to rely on the schema of change events themselves (inline JSON schema, or Avro schema in a registry)? Not all connectors provide the separate schema history topic, you won't have to parse DDL and the synchronization issue is avoided at all. Happy to discuss in more detail.

Disclaimer: I'm the Debezium lead.

Re: Materialize: A Streaming Data Warehouse

#102
post #51

Earlier quoted context omitted.

Indeed, Materialize is quite similar to Noria, and has the Frank McSherry stamp of awesomeness. [0] We know many of the Noria folks and have a lot of respect for them and their work. I also worked on the Noria project for a summer in college, and am a full-time engineer at Materialize now. The biggest difference is one of intended use. Noria is, first and foremost, a research prototype, intended to explore new ideas…

Do you have an example of where one might use WITH RECURSIVE?

A classical example is graph reachability. You'd express reachability via WITH RECURSIVE stating that nodes are reachable if there is a path of arbitrary length between them. (Recursion is needed since a plain SQL query can only query for paths up to a fixed length).
Post reply on HN