71–80 of 81 posts
Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
We're trying to solve this problem at ergodic.ai, combining structured tables and pdfs into a single KG
Earlier quoted context omitted.
This has always been the Hard Problem. For one, constructing an ontology that is comprehensive, flexible, and stable is huge effort. Then, taking the unstructured mess of documents and categorizing them is an entire industry in itself. Librarians have cataloging as a sub-specialty of library sciences devoted to this. So yes, there's a huge pile of tools and software for working with knowledge graphs, but to date popu…
When you boil it down, the current LLMs could work effectively if a prompt engineer could figure out a converging loop of a librarian tasked with generating a hypertext web ring crossed with a wikipedia. Perhaps one needs to manually create a starting point then ask the LLM to propse links to various documents or follow an existing one. Sufficiently loopable transversal should create a KG
Earlier quoted context omitted.
Can you expand on that? Where do big enterprise orgs products fit in, eg Microsoft, Google? What are the leading providers as you see them? As an outsider it is bewildering. First I hear that llama_index is good, then I hear that its overcomplicating slop. What sources or resources are reliable on this? How can we develop anything that will still stand in 12 months time?
May help to think of these tools as on the opposite end of the spectrum. As an analogy: 1. langchain, llamaindex, etc are the equivalent of jquery or ORMs for calling third-party LLMs. They're thin adapter layers with a bit of consistency and common tasks across. Arguably like React, where they are thin composition layers. So complaints of being leaky abstractions is in the sense of an ORM getting in the way vs helpi…
Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
You may want to take a look at Graphiti, which accepts plaintext or JSON input and automatically constructs a KG. While it’s primarily designed to enable temporal use cases (where data changes over time), it works just as well with static content. https://github.com/getzep/graphiti I’m one of the authors. Happy to answer any questions.
This becomes a cyclical hallucination problem. The LLM hallucinates and create incorrect graph which in turn creates even more incorrect knowledge.
We are working on this issue of reducing hallucination in knowledge graphs and using LLM is not at all the right way.
Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
You may want to take a look at Graphiti, which accepts plaintext or JSON input and automatically constructs a KG. While it’s primarily designed to enable temporal use cases (where data changes over time), it works just as well with static content. https://github.com/getzep/graphiti I’m one of the authors. Happy to answer any questions.
Earlier quoted context omitted.
There are lots of ways to go about RAG, many do not require graphs at all. I recommend looking at some simple spark queries to get an idea of what’s happening. What I’ve seen is using LLMs to identify what possible relationships some information may have by comparing it to the kinds of relationships in your database. Then when building the spark query it uses those relationships to query relevant data. The llm never…
i'm talking about a knowledge graph that explicitly stores data (=knowledge) as a graph and the question is how this solution establishes the connection to the llm. so that the llm uses the data ... anyway, never mind :)
Retrieving one with low latency is another.
Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
Came here to say this and glad I am not the only one. Building out an ontology seems like quite an expensive process. It would be hard to convince my stakeholders to do this.
I’ve had good success with CIM for Utilities to build a network graph for modelling the distribution and transmission networks adding sensor and event data for monitoring and analysis about 15 years ago.
Anywhere there is a technology focussed consortium of vendors and users building standards you will likely find a prebuilt graph. When RDF was “hot” many of the these groups spun out some attempt to model their domain.
In summary, if you need one look for one. Maybe there’s one waiting for you and you get to do less convincing and more doing.