Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grafeo – A fast, lean, embeddable graph database built in Rust

grafeo.dev

Post reply on HN
Print view View original

11–20 of 106 posts

Re: Grafeo – A fast, lean, embeddable graph database built in Rust

#11

Post by adsharma »

justonceokay wrote: post #9
jandrewrogers wrote: post #7

Earlier quoted context omitted.

That is a lot of code for what appears to be a vanilla graph database with a conventional architecture. The thing I would be cautious about is that graph database engines in particular are known for hiding many sharp edges without a lot of subtle and sophisticated design. It isn't obvious that the necessary level of attention to detail has been paid here.

Yes a graph database will happily lead you down a n^3 (or worse!) path when trying to query for a single relation if you are not wise about your indexes, etc.

Are you talking about the query plan for scanning the rel table? Kuzu used a hash index and a join.

Trying to make it optional.

Try

explain match (a)-[b]->(c) return a.rowid, b.rowid, c.rowid;

Re: Grafeo – A fast, lean, embeddable graph database built in Rust

#13

Post by arthurjean »

Aurornis wrote: post #4

Does anyone have any experience with this DB? Or context about where it came from? From the commit history it's obvious that this is an AI coded project. It was started a few months ago, 99% of commits are from 1 contributor, and that 1 contributor has some times committed 100,000 lines of code per week. (EDIT: 200,000 lines of code in the first week) I'm not anti-LLM, but I've done enough AI coding to know that one…

Sounds about right for someone who ships fast and iterates. 54 days for a v0 that probably needs refactoring isn't that crazy if the dev has a real DB background. We've all seen open source projects drag on for 3 years without shipping anything, that's not necessarily better

Re: Grafeo – A fast, lean, embeddable graph database built in Rust

#15

Post by adsharma »

adsharma wrote: post #10
jandrewrogers wrote: post #7

Earlier quoted context omitted.

That is a lot of code for what appears to be a vanilla graph database with a conventional architecture. The thing I would be cautious about is that graph database engines in particular are known for hiding many sharp edges without a lot of subtle and sophisticated design. It isn't obvious that the necessary level of attention to detail has been paid here.

Are you talking about Andy Pavlo bet here? https://news.ycombinator.com/item?id=29737326 Kuzu folks took some of these discussions and implemented them. SIP, ASP joins, factorized joins and WCOJ. Internally it's structured very similar to DuckDB, except for the differences noted above. DuckDB 1.5 implemented sideways information passing (SIP). And LadybugDB is bringing in support for DuckDB node tables. So the idea t…

Source: https://www.theregister.com/2023/03/08/great_graph_debate_we...

> There are some additional optimizations that are specific to graphs that a relational DBMS needs to incorporate: [...]

This is essentially what Kuzu implemented and DuckDB tried to implement (DuckPGQ), without touching relational storage.

The jury is out on which one is a better approach.

Re: Grafeo – A fast, lean, embeddable graph database built in Rust

#16

Post by jandrewrogers »

adsharma wrote: post #10
jandrewrogers wrote: post #7

Earlier quoted context omitted.

That is a lot of code for what appears to be a vanilla graph database with a conventional architecture. The thing I would be cautious about is that graph database engines in particular are known for hiding many sharp edges without a lot of subtle and sophisticated design. It isn't obvious that the necessary level of attention to detail has been paid here.

Are you talking about Andy Pavlo bet here? https://news.ycombinator.com/item?id=29737326 Kuzu folks took some of these discussions and implemented them. SIP, ASP joins, factorized joins and WCOJ. Internally it's structured very similar to DuckDB, except for the differences noted above. DuckDB 1.5 implemented sideways information passing (SIP). And LadybugDB is bringing in support for DuckDB node tables. So the idea t…

I wasn't referring to the Pavlo bet but I would make the same one! Poor algorithm and architecture scalability is a serious bottleneck. I was part of a research program working on the fundamental computer science of high-scale graph databases ~15 years ago. Even back then we could show that the architectures you mention couldn't scale even in theory. Just about everyone has been re-hashing the same basic design for decades.

As I like to point out, for two decades DARPA has offered to pay many millions of dollars to anyone who can demonstrate a graph database that can handle a sparse trillion-edge graph. That data model easily fits on a single machine. No one has been able to claim the money.

Inexplicably, major advances in this area 15-20 years ago under the auspices of government programs never bled into the academic literature even though it materially improved the situation. (This case is the best example I've seen of obviously valuable advanced research that became lost for mundane reasons, which is pretty wild if you think about it.)

Re: Grafeo – A fast, lean, embeddable graph database built in Rust

#19

Post by piyh »

gdotv wrote: post #5
Aurornis wrote: post #4

Does anyone have any experience with this DB? Or context about where it came from? From the commit history it's obvious that this is an AI coded project. It was started a few months ago, 99% of commits are from 1 contributor, and that 1 contributor has some times committed 100,000 lines of code per week. (EDIT: 200,000 lines of code in the first week) I'm not anti-LLM, but I've done enough AI coding to know that one…

Agreed, there's been a literal explosion in the last 3 months of new graph databases coded from scratch, clearly largely LLM assisted. I'm having to keep track of the industry quite a bit to decide what to add support for on https://gdotv.com and frankly these days it's getting tedious.

I'm turning off my brain and using neo4j
Post reply on HN