Why MySQL and not PostgreSQL or Redis on the roadmap for sources?
Show HN: Neum AI – Open-source large-scale RAG framework
21–30 of 31 posts
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#22Interesting to see that the semantic chunking in the tools library is a wrapper around GPT-4. Asks GPT for the python code and executes it: https://github.com/NeumTry/NeumAI/blob/main/neumai-tools/neu...
You seem to have found a very polite way of highlighting this.
I assume that in a not so distant future, a malware scanner will detect this and disallow one to run this locally.
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#23Why MySQL and not PostgreSQL or Redis on the roadmap for sources?
Postgres is already available :)
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#24Re: Show HN: Neum AI – Open-source large-scale RAG framework
#25If someone is about to start their project using Haystack would you suggest they instead look at Neumtry?
Well, of course I'm biased on the answer :). But to give a not-so-biased answer, I would first try to understand what the project is about and whether RAG is a priority in it. If the project is leveraging agents and LLMs without worrying too much on context/up-to-date data then Haystack could be a good option. If the focus is to eventually use RAG then our framework could help. Additionally, there might be a potentia…
Actually Haystack is very focused on RAG lately, just have a look at the latest blog articles: https://haystack.deepset.ai/blog
(Disclaimer: I am a Haystack maintainer)
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#26How is this any different from LlamaIndex [1]?
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#27Very bearish on these frameworks and abstractions.
Yes, obviously useful for prototyping and creating hype articles & tweets with fun examples. However any engineer is capable of doing their own rag with the same effort (minimal data extraction using the ancient pdf/scrape tools that are still open sota, or use cloud ocr for best —-> brute force chunking —-> embed —-> load in Ann with complementary metadata store)
Anyone doing prod needs to know the intricacies and make advanced engineering decisions. There’s a reason there aren’t similar end-to-end abstractions over creating Lucene (solr/elastic) indexes. Hmm, why not after many decades? …
In reality, the RAG tech is not entirely novel— it’s etl. Which in reality, complex etl is often a serious data curation effort. LLMs are the closest thing to enabling better data curation, and as long as you aren’t competing with open ai (arguably any commercial system is) then you can use chatgpt to create your chunks.
Beyond this embedding strategies are nice to abstract but the best approach to embeddings still remains to create your own and figure out contextual integration on your own. Creating your own can also just be fine-tuning. Inference is often an ensemble depending on your use case.
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#28Very bearish on these frameworks and abstractions. Yes, obviously useful for prototyping and creating hype articles & tweets with fun examples. However any engineer is capable of doing their own rag with the same effort (minimal data extraction using the ancient pdf/scrape tools that are still open sota, or use cloud ocr for best —-> brute force chunking —-> embed —-> load in Ann with complementary metadata store) An…
The problem is that business people have been just rubbing together libraries for decades making money (though maybe not the MOST) and will see these frameworks as a "simple" way to accelerate development, when in fact most of them are opinionated (and bad) ETL. Just put langchain together and it's done, right?
I went through building a RAG pipeline for a company and brought up at each stage how there's been no tuning, no efficacy testing for different scenarios, no testing of different chunking strategies, just the most basic work done and they released it almost immediately. Surprisingly to not much fan fare.
It doesn't really matter
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#29Earlier quoted context omitted.
Well, of course I'm biased on the answer :). But to give a not-so-biased answer, I would first try to understand what the project is about and whether RAG is a priority in it. If the project is leveraging agents and LLMs without worrying too much on context/up-to-date data then Haystack could be a good option. If the focus is to eventually use RAG then our framework could help. Additionally, there might be a potentia…
Actually Haystack is very focused on RAG lately, just have a look at the latest blog articles: https://haystack.deepset.ai/blog (Disclaimer: I am a Haystack maintainer)
A bit odd that they might not be aware of that. Any ideas from this project you see that might benefit Haystack?
Re: Show HN: Neum AI – Open-source large-scale RAG framework
#30How is this any different from LlamaIndex [1]? [1] https://www.llamaindex.ai
LlamaIndex is pretty awesome.
There are a couple areas where we think we are driving some differentiation.
1. The management of metadata as a first class citizen. This includes capturing metadata at every stage of the pipeline.
2. Be infra ready. We are still evolving this point, but we want to add abstractions that can help developers apply this type of framework to a large scale distributed architecture.
3. Enable different types of data synchronization natively. So far we enable both full and delta syncs, but have work in the pipeline to bring in abstractions for real-time syncing. 3.