Live data from Hacker News

A type-safe, realtime collaborative Graph Database in a CRDT

codemix.com

21–30 of 50 posts

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#21
post #9

Earlier quoted context omitted.

It's interesting to think of where the value comes from. Afaict 2 interesting areas: A: One of the main lessons of the RAG era of LLMs was reranked multiretrieval is a great balance of test time, test compute, and quality at the expense of maintaining a few costly index types. Graph ended up a nice little lift when put alongside text, vector, and relational indexing by solving some n-hop use cases. I'm unsure if the…

I'm conceptually very bullish on B (entity resolution and hierarchy pre-processing during writes). I'm less certain than A and B need to be merged into a single library. Obviously, a search agent should know the properties of the KG being searched, but as the previous poster mentioned, these graph dbs are inherently inaccurate and only form part of the retrieval pattern anyway.

Maybe it's useful to split out B1) KG pipelines from the choice of B2) simple property graph ontologies & queries vs advanced rdf ontologies and sparql queries

It sounds like you are thinking about KG pipelines, but I'm unclear on whether typed property graphs, vs more advanced RDF/SPARQL, is needed in your view on the graph engine side?

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#22
post #6

Can anyone explain why it is a good idea to make a graphdb in typescript? This not a language flamewar question, more of an implementation details question. Though typescript is pretty fast, and the language is flexible, we all know how demanding graph databases are. How hard they are to shard, etc. It seems like this could be a performance trap. Are there successful rbdms or nosql databases out there written in type…

I needed it to be possible to run the graph in the browser and cloudflare workers, so TS was a natural fit here. It was built as an experiment into end to end type safety - nothing to do with LLMs, but it ended up being useful in the product I'm building. It's not designed for large data sets.

You can use webassembly in both places https://developers.cloudflare.com/workers/runtime-apis/webas...

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#23

Cypher-over-Gremlin is a smart call — LLMs can write Cypher, makes the MCP angle viable in a new way. How dos Yjs handle schema migrations? If I add a property to a vertex type that existing peers have cached, does it conflict or drop the unknown field?

Instances that have not updated to the latest version of the schema will ignore the additional property but will not break or conflict.

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#24
I'm not terribly familiar with graph databases, but perhaps someone who is can explain the advantage of this awfully complicated seeming design. There's gremlin, cypher, yjs, and zod, all of which I understand are different languages for different problems.

What's the advantage of using all these different things in one system? You can do all of this in datalog. You get strong eventual consistency naturally. LLMs know how to write it. It's type safe. JS implementations exist [0].

[0] https://github.com/tonsky/datascript

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#25
post #6

Can anyone explain why it is a good idea to make a graphdb in typescript? This not a language flamewar question, more of an implementation details question. Though typescript is pretty fast, and the language is flexible, we all know how demanding graph databases are. How hard they are to shard, etc. It seems like this could be a performance trap. Are there successful rbdms or nosql databases out there written in type…

I needed it to be possible to run the graph in the browser and cloudflare workers, so TS was a natural fit here. It was built as an experiment into end to end type safety - nothing to do with LLMs, but it ended up being useful in the product I'm building. It's not designed for large data sets.

Gleam might be a great choice perhaps. Compiles to typescript and Erlang/BEAM.

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#27

I'm not terribly familiar with graph databases, but perhaps someone who is can explain the advantage of this awfully complicated seeming design. There's gremlin, cypher, yjs, and zod, all of which I understand are different languages for different problems. What's the advantage of using all these different things in one system? You can do all of this in datalog. You get strong eventual consistency naturally. LLMs kno…

Gremlin-like API gives end to end type safety if you're querying the database from TypeScript. This was the original motivation for the library.

Zod/Valibot/ArkType/Standard Schema support because you need a way to define your schema and this allows for that at runtime and compile time.

Y.js as a backing store because I needed to support offline sync, branching/forking, and I use Y.js for collaborative editing in my product, so I needed to be able to store the various CRDT types as properties within the graph. e.g. you can have a `description` property on your vertices or edges that is backed by a Y.Text or Y.XmlElement

Cypher because until the arrival of codemode it wasn't feasible to have LLMs write queries using the Gremlin-like API and LLMs already know Cypher.

Most of all though, this was an experiment that ended up being useful.

Re: A type-safe, realtime collaborative Graph Database in a CRDT

#29

I'm not terribly familiar with graph databases, but perhaps someone who is can explain the advantage of this awfully complicated seeming design. There's gremlin, cypher, yjs, and zod, all of which I understand are different languages for different problems. What's the advantage of using all these different things in one system? You can do all of this in datalog. You get strong eventual consistency naturally. LLMs kno…

The advantage for property graph databases using Cypher query language is that the queries for things like "show me all systems connected to this system by links greater than 10Gbps up to n hops away" are vastly easier to write and faster to complete compared to SQL and relational databases. Cypher lets you easily search for arbitrary graph patters and the result is also a graph, not a denormalized table.
Post reply on HN