Live data from Hacker News

Ask HN: What's your experience with using graph databases for agentic use-cases?

news.ycombinator.com

11–20 of 56 posts

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#11
Graph databases are one of those things that sound neat but you'll be hard pressed to find people using them that don't regret it.

I memorably had a job interview which consisted almost entirely of their senior architect going over exactly why he regretted introducing Neo4J several years earlier and how all the work is really about getting away from it. That was just the most extreme example.

The truth that people here don't like is that the Couch/Mongo style document DB is far more compelling as a intermediate point of structured/unstructured. There was even a mongo DB compatibility layer for foundation DB, but it doesn't seem to be maintained sadly. https://github.com/FoundationDB/fdb-document-layer

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#12
I'm building https://www.ergodic.ai - and we are using a graphs as the primary objects in which the intelligence operates.

I don't think every graph needs a graph database. For 99% of use-cases a relational database is the preferred solution to store a graph: provided that we have objects and ways to link objects, we're good to go. The advantages of graph dbs are in running more complex graph algorithms whenever that is required (transversal, etc) which is more efficient than "hacking it" with recursive queries in a relational db.

For us, I've yet to find the need for a dedicated graph db with few exceptions, and in those exceptions https://kuzudb.com/ was the perfect solution.

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#13
I had one client who made their whole thing about knowledge graphs, which I worked on because I needed money and it was interesting, but I am still a little suspicious that they may have had "knowledgebase" and "knowledge graphs" mixed up and did not know about vector search.

I think for the particular use case, something like filtering the vector search based on tags for each document and then (maybe) a relatively inexpensive and fast reranking LLM step could have worked as well or better. But the reranker is not necessarily important with a strong model and including enough results.

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#15
post #7

Graph DBs model relationship between entities. Is that a useful property of your retrieval task? They won't magically make your retrieval better without other additional work. How are you evaluating your current retrieval? Can you get to the point where you can compare your current solution with a Graph based one? A lot of the time i've seen people reach for a Graph DB they actually wanted/needed re-ranking of result…

>> An aside but the Director of ML at a company I worked for kept telling us "We need a Graph! We need a Graph!"

It depends. Maybe they knew something that the team didn't but couldn't articulate it. Maybe it would have been great. Alternately (and this seems to be a common tactic unfortunately), is they don't really know what they are doing but use the strategy of introducing a large / time consuming change and promise incredible things once the change is complete. The longer the change takes, the better in this situation as they can just chill while the change is taking place and polish resume for the next gig if it doesn't work out. If they jump to a new job before failure is obvious they can claim that they affected some large change at previous company and repeat the process. The other strategy is to performtatively claim success in the face of failure and move on to the next big thing.

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#16
The Terminus DB folks have been doing projects with LLM:s for some years by now, I'd assume they've had some success.

Don't remember if their licensing is annoying, but it was rather neat as a graph storage when I tried it out last christmas or thereabouts. If you actually have a fitting need it's probably a decent option, there are some graphical interfaces and so on that people who aren't technical specialists can use.

https://terminusdb.org/

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#17

Graph databases are one of those things that sound neat but you'll be hard pressed to find people using them that don't regret it. I memorably had a job interview which consisted almost entirely of their senior architect going over exactly why he regretted introducing Neo4J several years earlier and how all the work is really about getting away from it. That was just the most extreme example. The truth that people he…

>The truth that people here don't like is that the Couch/Mongo style document DB is far more compelling as a intermediate point of structured/unstructured.

in my opinion Graph DBs should only be used for highly structured data, which after all is what a graph is. Generally anything you would represent in SQL with too many joins to do queries you commonly have to do.

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#18

I'm building https://www.ergodic.ai - and we are using a graphs as the primary objects in which the intelligence operates. I don't think every graph needs a graph database. For 99% of use-cases a relational database is the preferred solution to store a graph: provided that we have objects and ways to link objects, we're good to go. The advantages of graph dbs are in running more complex graph algorithms whenever that…

> For 99% of use-cases a relational database is the preferred solution…

After enough years you realize this is the case for every single problem

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#19

I'm building https://www.ergodic.ai - and we are using a graphs as the primary objects in which the intelligence operates. I don't think every graph needs a graph database. For 99% of use-cases a relational database is the preferred solution to store a graph: provided that we have objects and ways to link objects, we're good to go. The advantages of graph dbs are in running more complex graph algorithms whenever that…

> which is more efficient than "hacking it" with recursive queries in a relational db

It seems to me that the way recursive CTEs were originally defined is the biggest reason that relational databases haven't been more successful with users who need to run serious graph workloads - in Frank McSherry's words:

> As it turns out, WTIH RECURSIVE has a bevy of limitations and mysterious semantics (four pages of limitations in the version of the standard I have, and I still haven't found the semantics yet). I certainly cannot enumerate, or even understand the full list [...] There are so many things I don't understand here.

https://github.com/frankmcsherry/blog/blob/master/posts/2022...

Re: Ask HN: What's your experience with using graph databases for agentic use-cases?

#20
Graph RAG is great, but misunderstood.

As a human user, consider file-system navigation or code search. You navigate to a seed file, and then hop through the children and dependencies until you find what you're looking for. The value is in explicit and direct edges. Perfect search is hard. Landing in the right neighborhood is less so. (It's like golf if you think about it)

Agentic systems loop through the following steps - (Plan -> Inquire -> Retrieve -> Observe -> Act -> Repeat). The agent interacts with your search-system during the inquire and retrieve phases. In these phrases, There are 2 semantic problems that a simple embedding based search or a simple db alone can't solve: seeding and completeness. Seeding - How do you ask a good question when you don't know what you don't know ? Completeness - once you know a little bit, how do you know that you have obtained everything you need to answer a question ?

A solid embedding based search allows under-defined free-form inquiry, and puts the user near the data they're looking for. From there, an explicit graph allows the agent to navigate through the edges until it hits gold or gives the agent enough signal to retry with better informed free-form inquiry. Together, they solve the seeding problem. Now, once you have found a few seed nodes to work off of, the agent can keep exploring the neighbors, until they become sufficiently irrelevant. At that threshold, the retrieval system can return the explored nodes with a measurable metric of confidence in completeness. This makes completeness a measure that you can optimize, helping solve the 2nd problem.

You'll notice that there is no magic here. The quality of your search will depend on the quality of your edges, entities, exploration strategy and relevance detectors. This requires a ton of hand-engineering and subject specific domain knowledge, neither of which are systems bottlenecks. The data-store itself will do very little to help get you a better answer.

Which brings me to your question, the datastore. The datastore only matters at sufficient scale. You CAN implement Graph RAG in a standard database. Get a column to track your edges, a column to track entities and some way to search over embeddings and you're good. You can get it done in an afternoon (until permissions become an issue, but I digress).

We know that a spotlight style file-system search works just fine on 100k+ documents, while your mac's fan barely even turns on. If you're asking this question, then your company probably doesn't scale past that point. In fact, I'd argue that few companies will ever cross that threshold for agentic operations. At this scale, your postgres instance won't be the bottleneck.

Comparing postgres to graph-rag-startups, the real value of using a native graph-RAG solution is their defaults. The companies know that their user's need is agentic semantic search, and the products come preloaded with defaults that give you embeddings, entities and graph-edges that aren't completely useless. From a practical standpoint, those extras might push you over the edge. But be aware that your performance gains are coming from outsourcing the hand-engineering of features and not the data structure itself.

My personal opinion is to keep the data structure as simple as possible. MLEs and Data Scientists are mediocre systems engineers and it is okay to accept that. You want your ML & product team to be able to iterate on the search-logic and quality as fast as possible. That's where the real gains will come from. Speaking from experience, premature optimization in a new field will slow your team down to a crawl. IE. Go with postgres if that's what's simple for everyone to work with.

tldr: It's not about the scalability of the datastructure, it's about how you use it.

Post reply on HN