Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Using Unstructured Data and Role of Knowledge Graphs

kuzudb.com

Post reply on HN
Print view View original

21–30 of 32 posts

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#21

Post by Der_Einzige »

dmezzetti wrote: post #10

If you're interested in graphs + RAG and want an alternate approach, txtai has a semantic graph component. https://neuml.hashnode.dev/introducing-the-semantic-graph https://github.com/neuml/txtai Disclaimer: I'm the primary author of txtai

Note for those who aren't aware, a "Semantic Graph" means a knowledge graph built using a "sentence(pooled) transformer" language model to draw edges between the vertices (text data at whatever granularity the user decides) according to semantic similarity.

What's awesome about them is that they essentially form in my mind the "extractive" analogue to LLMs "generative" nature.

Semantic Graphs give every single graph theory algorithm a unique epistemological twist given any particular dataset. In my case, I've built and released pre-trained semantic graphs for my debate evidence. I observe that path traversals form "debate cases", and that graph centrality in this case finds the most "generic/universally applicable" evidence. Given a different dataset, the same algorithms will have different interpretations.

What makes txtai so awesome is that it creates a synchronized interface between an underlying vector DB, SQL DB, and a semantic knowledge graph. The flexibility and power this offers compared to other vector DB solutions is simply unparalleled. I have seen zero meaningful competition from a vectorDB industry which is flooded with money despite little product differentiation among themselves.

Disclaimer: I wrote an NLP paper with dmezzetti as my co-author about semantic graphs: https://aclanthology.org/2023.newsum-1.10.pdf

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#22

Post by dmezzetti »

bryan0 wrote: post #18
dmezzetti wrote: post #10

If you're interested in graphs + RAG and want an alternate approach, txtai has a semantic graph component. https://neuml.hashnode.dev/introducing-the-semantic-graph https://github.com/neuml/txtai Disclaimer: I'm the primary author of txtai

This is really cool, I'm surprised I never heard of this project before. The examples look really clean. Most RAG tools seem to start with the LLM and add Vector building and retrieval around it, while this tool seems like it started with Vector / Graph building and retrieval, then added LLM support later.

Thanks, that's an accurate assessment. The main reason for this approach is that txtai has been around since 2020 before the LLM era.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#23

Post by dmezzetti »

Der_Einzige wrote: post #21
dmezzetti wrote: post #10

If you're interested in graphs + RAG and want an alternate approach, txtai has a semantic graph component. https://neuml.hashnode.dev/introducing-the-semantic-graph https://github.com/neuml/txtai Disclaimer: I'm the primary author of txtai

Note for those who aren't aware, a "Semantic Graph" means a knowledge graph built using a "sentence(pooled) transformer" language model to draw edges between the vertices (text data at whatever granularity the user decides) according to semantic similarity. What's awesome about them is that they essentially form in my mind the "extractive" analogue to LLMs "generative" nature. Semantic Graphs give every single graph…

Thank you for taking the time to share these excellent additional details!

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#24

Post by Keyframe »

softwaredoug wrote: post #5

When I started working in search 10+ years ago, people would build a beautiful UI, and then, only on shipping, realize the search results were trash + irrelevant. They imagined a search system like Elasticsearch was basically Google. When in reality, Elasticsearch is just a bit of infrastructure. A framework, not a solution. There's a similar thing happening on RAG. Where people think building the chat interaction is…

Great observation. I've seen it often in tech, across the board. It's no better, maybe a step up, than 'idea guy' who 'just' needs someone to build his idea. Hand-waving or complete lack of awareness on the actual value (hard) part.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#25

Post by semihsalihoglu »

laminarflow027 wrote: post #17
formercoder wrote: post #4

It’s interesting to see more developed KG + LLM use cases that aren’t just NL to Graph DB Query Lang.

Totally agree! The wave of blog posts and examples one sees where it's just text-to-SQL or text-to-Cypher or any other query lang aren't really exploring the topic at any level of technical depth, and we need to see more evaluations and technical papers that characterize them, so that we can understand how to build better systems.

I think even on the LLMS + KGs space the depth is not very deep. In fact there is more technical depth in the text-to-SQL than anything else I have seen on LLMs. Maybe the COLBERT-like matrix-models is another topic where there is good technical depth.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#26

Post by hobs »

softwaredoug wrote: post #5

When I started working in search 10+ years ago, people would build a beautiful UI, and then, only on shipping, realize the search results were trash + irrelevant. They imagined a search system like Elasticsearch was basically Google. When in reality, Elasticsearch is just a bit of infrastructure. A framework, not a solution. There's a similar thing happening on RAG. Where people think building the chat interaction is…

I spent 8 months telling people this before I got laid off while the CEO continues to chase LLM money with no new ideas or even the talent to solve the problem.

They spent so much time on the UI and basically left the actual search to the last minute, and it was a hilarious failure on launch.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#27

Post by dbish »

softwaredoug wrote: post #5

When I started working in search 10+ years ago, people would build a beautiful UI, and then, only on shipping, realize the search results were trash + irrelevant. They imagined a search system like Elasticsearch was basically Google. When in reality, Elasticsearch is just a bit of infrastructure. A framework, not a solution. There's a similar thing happening on RAG. Where people think building the chat interaction is…

Yep, we're doing RAG-ish search and ranking across many context types and modalities, you definitely can't just use a vectordb and do some chunking/search, there are a wide variety of search-like ranking, clustering, etc. and domain specific work for relevance and it's very hard to measure and prove improvements.

It's going to just evolve into recreating the various search and ranking processes of old just on top of a bit more semantic understanding with some smarter NLG layered in :). It won't be just LLMs, we'll have intent classification, named entity recognition, a personalization layer, reranking, all that fun stuff again.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#28

Post by hackernoteng »

softwaredoug wrote: post #5

When I started working in search 10+ years ago, people would build a beautiful UI, and then, only on shipping, realize the search results were trash + irrelevant. They imagined a search system like Elasticsearch was basically Google. When in reality, Elasticsearch is just a bit of infrastructure. A framework, not a solution. There's a similar thing happening on RAG. Where people think building the chat interaction is…

This is a great comment. Good search is really hard. RAG is much harder. At least with search user can pick the best result manually or refine their search. With RAG you pass topK to the LLM and assume its good results. The assumption is that its "semantic search" with vectors so it will just work... wrong.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#29

Post by moralestapia »

softwaredoug wrote: post #5

When I started working in search 10+ years ago, people would build a beautiful UI, and then, only on shipping, realize the search results were trash + irrelevant. They imagined a search system like Elasticsearch was basically Google. When in reality, Elasticsearch is just a bit of infrastructure. A framework, not a solution. There's a similar thing happening on RAG. Where people think building the chat interaction is…

Hmm, RAG is not "the chat interaction", that's GPT or any other "brain" you choose.

Last week I finished building my 3rd RAG stack for legal document retrieval. Almost-vanilla RAG got me 90-95% of the way. Only drawback is cost, still 10x-100x above the ideal price point; but that will only improve in the future.

Re: RAG Using Unstructured Data and Role of Knowledge Graphs

#30

Post by sroussey »

laminarflow027 wrote: post #15
sroussey wrote: post #12

Earlier quoted context omitted.

What open source model is good at pulling keyterms?

OpenNRE ( https://github.com/thunlp/OpenNRE ) is another good approach to neural relation extraction, though it's slightly dated. What would be particularly interesting is to combine models like OpenNRE or SpanMarker with entity-linking models to construct KG triples. And a solid, scalable graph database underneath would make for a great knowledge base that can be constructed from unstructured text.

Nice, I’ll look that up.

I was thinking in terms of RAG and turning text into keywords. Any thoughts there?

Post reply on HN