Nice! Could you elaborate on "not just a basic RAG"?
Thank you!
Typical RAG implementations I’ve seen take the user query and directly run it against the full-text search and embedding indexes. This produces sub-par results because the query embedding doesn’t really capture fully what the user is really looking for.
A better solution is to send the user query to the LLM, and let it construct and run queries against the index via tool calling. Nothing too ground-breaking tbh, pretty much every AI search agent does this now. But it produces much better results.
Tangeant: Why is integrating with teams SO difficult?
I started parsing its system logs to create entries in our system automatically to book my times - just not todeal with their silly REST api requirements.
* "Self-hosted: Runs entirely on your infrastructure. No data leaves your network." * "Bring Your Own LLM: Anthropic, OpenAI, Gemini, or open-weight models via vLLM." With so many newbies wanting these kinds of services it might be worth adjusting the first bullet to say: "No data leaves your network, at least as long as you don't use any Anthropic, OpenAI, or Gemini models via the network of course"
Most organizations are going to be self hosting on aws, gcp or azure... So as long as you use their inference services as your LLM then you can keep it all within the private network
Exactly, enterprise customers almost always use private model endpoints on their cloud provider for any serious deployments. Data stays within the customer's VPC, data security and privacy is guaranteed by the cloud providers.
* "Self-hosted: Runs entirely on your infrastructure. No data leaves your network." * "Bring Your Own LLM: Anthropic, OpenAI, Gemini, or open-weight models via vLLM." With so many newbies wanting these kinds of services it might be worth adjusting the first bullet to say: "No data leaves your network, at least as long as you don't use any Anthropic, OpenAI, or Gemini models via the network of course"
That's a good point, it might make sense to clarify that for individuals who want to self-host. I'll make the change, thanks!
* "Self-hosted: Runs entirely on your infrastructure. No data leaves your network." * "Bring Your Own LLM: Anthropic, OpenAI, Gemini, or open-weight models via vLLM." With so many newbies wanting these kinds of services it might be worth adjusting the first bullet to say: "No data leaves your network, at least as long as you don't use any Anthropic, OpenAI, or Gemini models via the network of course"
Most organizations are going to be self hosting on aws, gcp or azure... So as long as you use their inference services as your LLM then you can keep it all within the private network
Even self-hosting on AWS, GCP, or Azure isn't local enough for certain application, such as people doing export-controlled work where any sysadmin or person with physical access to the server/data is required to be a US Person (or equivalent in other countries). This is the niche that the govcloud solutions are aimed at serving. But some people just want to build big actually-private, actually self-hosted systems and do their own physical and network security.
Can we please not change the meaning of chat to mean agent interface? It was painful to see crypto suddenly meaning token instead if cryptography. Plus i really dont want to “chat” with ai. its a textual interface
How does it compare to Onyx (rebranded from Danswer, with more chat focus, while Danswer was more RAG focus on company docs/comms)? - https://onyx.app/ - Their rebranded Onyx launch: https://news.ycombinator.com/item?id=46045987 - Their orignal Danswer launch: https://news.ycombinator.com/item?id=36667374
So far both projects are quite similar… the only major difference being the search index. Onyx uses vespa.ai for BM25 and vector search, I decided to go down the Postgres-only route.
Can we please not change the meaning of chat to mean agent interface? It was painful to see crypto suddenly meaning token instead if cryptography. Plus i really dont want to “chat” with ai. its a textual interface
Fair point, although I think we have OpenAI to blame for that - for buying chat.com and pointing it to the most popular textual AI interface of them all :)
Nice! Could you elaborate on "not just a basic RAG"?
Thank you! Typical RAG implementations I’ve seen take the user query and directly run it against the full-text search and embedding indexes. This produces sub-par results because the query embedding doesn’t really capture fully what the user is really looking for. A better solution is to send the user query to the LLM, and let it construct and run queries against the index via tool calling. Nothing too ground-breakin…