advice to OP - that gif showing how you zoom in and star the repo is a giant turnoff. i closed my tab when i saw that.
Agreed. Do you think potential users of your repo don't know how to star it?
KAG – Knowledge Graph RAG Framework
21–30 of 81 posts
Re: KAG – Knowledge Graph RAG Framework
#22a quick look leaves me with the question:
what exactly is being tokenized? RDS, OWL, Neo4j, ...?
how is the knowledge graph serialized?
Re: KAG – Knowledge Graph RAG Framework
#23LLMs are not that different from humans, in both cases you have some limited working memory and you need to fit the most relevant context into it. This means that if you have a new knowledge base for llms it should be useful for humans too. There should be a lot of cross pollination between these tools. But we need a theory on the differences too. Now it is kind of random how we differentiate the tools. We need ergon…
>ergonomics for LLMs
When I need to build something for an LLM to use, I ask the LLM to build it. That way, by definition, the LLM has a built in understanding of how the system should work, because the LLM itself invented it.
Similarly, when I was doing some experiments with a GPT-4 powered programmer, in the early days I had to omit most of the context (just have method stubs). During that time I noticed that most of the code written by GPT-4 was consistently the same. So I could omit its context because the LLM would already "know" (based on its mental model) what the code should be.
Re: KAG – Knowledge Graph RAG Framework
#24LLMs are not that different from humans, in both cases you have some limited working memory and you need to fit the most relevant context into it. This means that if you have a new knowledge base for llms it should be useful for humans too. There should be a lot of cross pollination between these tools. But we need a theory on the differences too. Now it is kind of random how we differentiate the tools. We need ergon…
>ergonomics for LLMs When I need to build something for an LLM to use, I ask the LLM to build it. That way, by definition, the LLM has a built in understanding of how the system should work, because the LLM itself invented it. Similarly, when I was doing some experiments with a GPT-4 powered programmer, in the early days I had to omit most of the context (just have method stubs). During that time I noticed that most…
> the LLM has a built in understanding of how the system should work, because the LLM itself invented it
Really? I’m not sure that the word “understanding” means the same thing to you as it does to me.
Re: KAG – Knowledge Graph RAG Framework
#25What do other HNers make out of this? Would you use this? Responsible for a legaltech startup here.
If you have to deal with domain specific data, then this would not work as well. I mean it will get you an incremental shift (based on what I see, it's just creating explicit relationships at the index time instead of letting the model do it at runtime before generating an output. Effective incrementally, but depends on type of data.) yes, though not enough to justify redoing your own pipeline. You are likely better off with your current approach and developing robust evals.
If you want a transformational shift in terms of accuracy and reasoning, the answer is different. Many a times RAG accuracy suffers because the text is out of distribution, and ICL does not work well. You get away with it if all your data is in public domain in some form (ergo, llm was trained on it), else you keep seeing the gaps with no way to bridge them. I published a paper around it and how to effciently solve it, if interested. Here is a simplified blog post on the same: https://medium.com/@ankit_94177/expanding-knowledge-in-large...
Edit: Please reach out here or on email if you would like further details. I might have skipped too many things in the above comment.
Re: KAG – Knowledge Graph RAG Framework
#26LLMs are not that different from humans, in both cases you have some limited working memory and you need to fit the most relevant context into it. This means that if you have a new knowledge base for llms it should be useful for humans too. There should be a lot of cross pollination between these tools. But we need a theory on the differences too. Now it is kind of random how we differentiate the tools. We need ergon…
> This means that if you have a new knowledge base for llms it should be useful for humans too. There should be a lot of cross pollination between these tools.
This is realistic but hence going to be unpopular unfortunately, because people expect magic / want zero effort.
Re: KAG – Knowledge Graph RAG Framework
#27Earlier quoted context omitted.
I enjoy the explosion of tools. Only time will tell which ones stand the test of time. But this is my day job so I never get tired of new tools but I can see how non-industry folks can find it overwhelming
Can you expand on that? Where do big enterprise orgs products fit in, eg Microsoft, Google? What are the leading providers as you see them? As an outsider it is bewildering. First I hear that llama_index is good, then I hear that its overcomplicating slop. What sources or resources are reliable on this? How can we develop anything that will still stand in 12 months time?
> How can we develop anything that will still stand in 12 months time?
The pace at which things are moving, likely none. You will have to keep making changes as and when you see newer things. One thing in your favor (arguably) is that every technique is very dependent on the dataset and problem you are solving. So, if you do not have the latest one implemented, you would be okay, as long as your evals and metrics are good. So, if this helps, skip the details, understand the basics, and go for your own implementation. One thing to look out for is new SOTA LLM releases, and the jumps in capability. Eg: 4o did not announce it, but they started doing very well on vision. (GPT-4 was okay, 4o is empirically quite better). These things help when you update your pipeline.
Re: KAG – Knowledge Graph RAG Framework
#28Fancy, I think, but again no word on the actual work of turning a few bazillion csv files and pdf's into a knowledge graph. I see a lot of these KG tools pop up, but they never solve the first problem I have, which is actually constructing the KG itself.
I have been building something like this for myself. Is there a room for a paid software, and would you be willing to pay for something like that?
Re: KAG – Knowledge Graph RAG Framework
#29"Whitepaper" is guarded behind this:
https://survey.alipay.com/apps/zhiliao/n33nRj5OV
> The white paper is only available for professional developers from different industries. We need to collect your name, contact information, email address, company name, industry type, position and your download purpose to verify your identity...
That's new.
Re: KAG – Knowledge Graph RAG Framework
#30LLMs are not that different from humans, in both cases you have some limited working memory and you need to fit the most relevant context into it. This means that if you have a new knowledge base for llms it should be useful for humans too. There should be a lot of cross pollination between these tools. But we need a theory on the differences too. Now it is kind of random how we differentiate the tools. We need ergon…
>ergonomics for LLMs When I need to build something for an LLM to use, I ask the LLM to build it. That way, by definition, the LLM has a built in understanding of how the system should work, because the LLM itself invented it. Similarly, when I was doing some experiments with a GPT-4 powered programmer, in the early days I had to omit most of the context (just have method stubs). During that time I noticed that most…
> the LLM has a built in understanding of how the system should work,
Thats not how an LLM works. It doesn't understand your question, nor the answer. It can only give you a statistically significant sequence of words that should follow what you gave it.