Live data from Hacker News

Drasi: Microsoft's open source data processing platform for event-driven systems

github.com

11–20 of 87 posts

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#11

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

I too have great memories of cypher. Such an elegant way to write queries.

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#12

Oh this very much reminds me of [feldera]( https://feldera.com ) — they do incremental loads and computations using some novel approaches (most of which i am too dumb to follow). Really nice folks too.

Or the related Materialize stuff https://materialize.com/

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#13

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

> while most of the queries could also be translated to relational SQL, Cypher's linear construction using WITH clauses is far, far easier to reason about, IMO.

https://prql-lang.org/

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#14
post #11

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

I too have great memories of cypher. Such an elegant way to write queries.

If you haven't been following it, I recently found out that it is now supported in a limited capacity by Google Spanner[0]. The openCypher initiative started a few years back and it looks like it's evolved into the (unfortunate moniker) GQL[1].

So it may be the case that we'll see more Cypher out in the wild.

[0] https://cloud.google.com/spanner/docs/graph/opencypher-refer...

[1] https://neo4j.com/blog/cypher-gql-world/

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#16

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

> while most of the queries could also be translated to relational SQL, Cypher's linear construction using WITH clauses is far, far easier to reason about, IMO. https://prql-lang.org/

Didn't look too deeply, but one of the keys with Cypher (at least in the context of graph databases) is that it has a nice way of representing `JOIN` operations as graph traversals.

    MATCH (p:Person)-[r]-(c:Company) RETURN p.Name, c.Name
Where `r` can represent any relationship (AKA `JOIN`) between the two collections `Person` and `Company` such as `WORKS_AT`, `EMPLOYED_BY`, `CONTRACTOR_FOR`, etc.

So I'd say that linear queries are one of the things I like about Cypher, but the clean abstraction of complex `JOIN` operations is another huge one.

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#17

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

We made a health backend partly using Cypher and the only thing I found was the simple queries looked amazing, but as soon as you need to join non-linearly it started looking a lot like SQL again. And when you're using an ORM it stops mattering. And when you need migrations it gets painful!

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#18

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

We made a health backend partly using Cypher and the only thing I found was the simple queries looked amazing, but as soon as you need to join non-linearly it started looking a lot like SQL again. And when you're using an ORM it stops mattering. And when you need migrations it gets painful!

    > but as soon as you need to join non-linearly
At least in our use case, even with some very gnarly 20+ line Cypher queries, it never got to the point where it felt like SQL and certainly, those same queries would be even gnarlier as nested sub-selects, CTEs, or recursive selects, IMO.

Perhaps a characteristic of our model (a taxonomy of Region, Country, Sponsor, Program, Trial, Site, Staff for global clinical trials and documents required by Region/Country/Program/Trial).

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#19

Very interesting choice of using Cypher[0] In 2014, we built a similar type event-driven system (but specifically for document distribution (a document can be distributed to a target set of entities; if a new entity is added, we need to resolve which distributions match)) and also ended up using Cypher via Neo4j (because of the complex taxonomical structure of how we mapped entities). It is a super underrated query l…

We made a health backend partly using Cypher and the only thing I found was the simple queries looked amazing, but as soon as you need to join non-linearly it started looking a lot like SQL again. And when you're using an ORM it stops mattering. And when you need migrations it gets painful!

"you need to join non-linearly "

What does this mean?

Re: Drasi: Microsoft's open source data processing platform for event-driven systems

#20

Earlier quoted context omitted.

We made a health backend partly using Cypher and the only thing I found was the simple queries looked amazing, but as soon as you need to join non-linearly it started looking a lot like SQL again. And when you're using an ORM it stops mattering. And when you need migrations it gets painful!

> but as soon as you need to join non-linearly At least in our use case, even with some very gnarly 20+ line Cypher queries, it never got to the point where it felt like SQL and certainly, those same queries would be even gnarlier as nested sub-selects, CTEs, or recursive selects, IMO. Perhaps a characteristic of our model (a taxonomy of Region, Country, Sponsor, Program, Trial, Site, Staff for global clinical trials…

Cypher works really well with a well defined taxonomy.
Post reply on HN