Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Neum AI – Open-source large-scale RAG framework

github.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#11

Post by eigenvalue »

picohen wrote: post #4
eigenvalue wrote: post #2

Cool. Do you do any of the relevance calculations directly, or is that all handled by Weaviate? If so, is there any way to influence that part of it, or is it something of a black box?

Relevance calculations are handled by the vector db but we try to improve such relevance with the use of metadata (you will see how our components have "selectors" so that metadata can flow all the way to the vector database at the vector level and have an influence when results/scores get retrieved at search time)

Got it. I'd encourage you to expose more of that functionality at the level of your application if possible. I think there is a lot of potential in using more than just cosine similarity, especially when there are lots of candidates and you really want to sharpen up the top few recommendations to the best ones. You might find this open-source library I made recently useful for that:

https://github.com/Dicklesworthstone/fast_vector_similarity

I've had good results from starting with cosine similarity (using FAISS) and then "enriching" the top results from that with more sophisticated measures of similarity from my library to get the final ranking.

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#13

Post by ddematheu »

hrpnk wrote: post #10

Interesting to see that the semantic chunking in the tools library is a wrapper around GPT-4. Asks GPT for the python code and executes it: https://github.com/NeumTry/NeumAI/blob/main/neumai-tools/neu...

Yeah, we were playing around with doing some semantic chunking. Works okay for some use cases. We have some ideas to go further on that.

Generally we have found that recursive chunking and character chunking tend to be short sighted.

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#14

Post by hrpnk »

ddematheu wrote: post #13
hrpnk wrote: post #10

Interesting to see that the semantic chunking in the tools library is a wrapper around GPT-4. Asks GPT for the python code and executes it: https://github.com/NeumTry/NeumAI/blob/main/neumai-tools/neu...

Yeah, we were playing around with doing some semantic chunking. Works okay for some use cases. We have some ideas to go further on that. Generally we have found that recursive chunking and character chunking tend to be short sighted.

Don't you find it dangerous to just run the code w/o any sanitizing?

Why not capture a few strategies that the LLM returns as code that can be properly audited (and ran locally improving the overall performance)?

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#15

Post by alchemist1e9 »

picohen wrote: post #7
alchemist1e9 wrote: post #5

If someone is about to start their project using Haystack would you suggest they instead look at Neumtry?

Well, of course I'm biased on the answer :). But to give a not-so-biased answer, I would first try to understand what the project is about and whether RAG is a priority in it. If the project is leveraging agents and LLMs without worrying too much on context/up-to-date data then Haystack could be a good option. If the focus is to eventually use RAG then our framework could help. Additionally, there might be a potentia…

I understood Haystack as doing RAG but your comment seems to define it differently than my understanding.

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#16

Post by ddematheu »

hrpnk wrote: post #14
ddematheu wrote: post #13

Earlier quoted context omitted.

Yeah, we were playing around with doing some semantic chunking. Works okay for some use cases. We have some ideas to go further on that. Generally we have found that recursive chunking and character chunking tend to be short sighted.

Don't you find it dangerous to just run the code w/o any sanitizing? Why not capture a few strategies that the LLM returns as code that can be properly audited (and ran locally improving the overall performance)?

It is dangerous, part of the reason that we haven't productized that further. One of the ideas we had to productize the capabilities further was to leverage edge / lambda functions to compartmentalize the code generated. (Plus it becomes a general extensibility for folks that are not using semantic code generation and simply want to write their own code.)

The idea of auditing the strategy is interesting. The flow that we have used for the semantic chunkers up to date has been along these lines where we : 1) Use the utility to generate the code snippets (and do some manual inspection) 2) Test the code snippets against some sample text 3) Validate the results

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#17

Post by treprinum »

ddematheu wrote: post #13
hrpnk wrote: post #10

Interesting to see that the semantic chunking in the tools library is a wrapper around GPT-4. Asks GPT for the python code and executes it: https://github.com/NeumTry/NeumAI/blob/main/neumai-tools/neu...

Yeah, we were playing around with doing some semantic chunking. Works okay for some use cases. We have some ideas to go further on that. Generally we have found that recursive chunking and character chunking tend to be short sighted.

Why not use Stanford Stanza?

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#18

Post by matmulbro »

this sums up current wave of AI 'companies':

submissions by this user (https://news.ycombinator.com/submitted?id=picohen):

Show HN: Neum AI – Open-source large-scale RAG framework (github.com/neumtry)

Show HN: ElectionGPT – easy-to-consume information about U.S. candidates (electiongpt.ai)

Efficiently sync context for your LLM application (neum.ai)

Show HN: Neum AI – Improve your AI's accuracy with up-to-date context (neum.ai)

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#19

Post by picohen »

matmulbro wrote: post #18

this sums up current wave of AI 'companies': submissions by this user ( https://news.ycombinator.com/submitted?id=picohen ): Show HN: Neum AI – Open-source large-scale RAG framework (github.com/neumtry) Show HN: ElectionGPT – easy-to-consume information about U.S. candidates (electiongpt.ai) Efficiently sync context for your LLM application (neum.ai) Show HN: Neum AI – Improve your AI's accuracy with up-to-date conte…

Gotta start somewhere! Take a look at this one as well! https://news.ycombinator.com/item?id=37824547

Re: Show HN: Neum AI – Open-source large-scale RAG framework

#20

Post by obiefernandez »

matmulbro wrote: post #18

this sums up current wave of AI 'companies': submissions by this user ( https://news.ycombinator.com/submitted?id=picohen ): Show HN: Neum AI – Open-source large-scale RAG framework (github.com/neumtry) Show HN: ElectionGPT – easy-to-consume information about U.S. candidates (electiongpt.ai) Efficiently sync context for your LLM application (neum.ai) Show HN: Neum AI – Improve your AI's accuracy with up-to-date conte…

I haven't yet seen any competitor come close to what we've achieved at my startup https://olympia.chat - very humanlike assistants crafted specifically for solopreneurs and bootstrapped startups
Post reply on HN