Build a chatbot with custom data sources, powered by LlamaIndex
blog.streamlit.io
Build a chatbot with custom data sources, powered by LlamaIndex
1–10 of 13 posts
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#2I'd love to set up one of those fancy chatbots for my almost dead IRC channel.
Does anyone know something out there?
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#3I'd love to set up one of those fancy chatbots for my almost dead IRC channel. Does anyone know something out there?
[deleted]
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#4A few scattered thoughts:
Firstly, the amalgamation of custom data sources via RAG pipelines sounds enticing on paper, but there are inherent complications. when fusing disparate data silos, considerations around data normalization, entropy reduction, and the potential amplification of noise-to-signal ratio are paramount. simply bolting on data to an LLM doesn't guarantee efficacy.
Then there's the underlying premise of using LLM as a primary UI model. while GPT and its ilk have made waves, a chatbot interface has inherent limitations. there's the conversational depth challenge, context retention gaps, and the non-trivial issue of multi-turn dialogues throwing off the model. Not to mention, the computational overhead of on-the-fly, real-time retrievals needs to be closely monitored to avoid a latency nightmare, especially if the knowledge base grows.
Further, there's a certain laziness in simply bolting on an LLM chatbot interface, which isn't appropriate for all settings, and can present accessibility challenges.
Speaking of the KB, the two-stage process delineated – indexing and querying – while conceptually sound, often buckles under scale. Memory networks and transformer architectures have their limits, and I'm curious about how LlamaIndex circumvents these bottlenecks.
also, leaning heavily on Streamlit could be a double-edged sword. Can anyone speak to its adaptability in production environments, especially under high concurrency?
anyway, I think it's important to bear in mind that chatbots and LLMs are tools, not magic wands or universal hammers
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#5I'd love to set up one of those fancy chatbots for my almost dead IRC channel. Does anyone know something out there?
Check out LLMStack (https://github.com/trypromptly/LLMStack). It allows you to build LLM apps and chatbots and provides you with an API for integration.
It should be possible to write a simple script to link LLMStack's app with your IRC channel. API documentation for running the apps is at https://llmstack.ai/docs/apis/introduction#apps. Let me know if you need help with this.
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#6A few scattered thoughts: Firstly, the amalgamation of custom data sources via RAG pipelines sounds enticing on paper, but there are inherent complications. when fusing disparate data silos, considerations around data normalization, entropy reduction, and the potential amplification of noise-to-signal ratio are paramount. simply bolting on data to an LLM doesn't guarantee efficacy. Then there's the underlying premise…
The data questions are exactly the things we are working through right now!
I think Streamlit is a great way to get started quickly. Would love to talk more about your thoughts around data ingest for prod use cases. yiding@runllama.ai
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#7RAG tools are usually very trivial w.r.t. extracting clean data for embeddings. Demos typically show clean data sources with orthogonal data which lends itself well to good performing embeddings based IR. Real-world data is mixed up, messy, and requires a lot of work to extract, clean, normalize, etc. Typically fine-tuned embedding model would be needed.
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#8RAG tools are usually very trivial w.r.t. extracting clean data for embeddings. Demos typically show clean data sources with orthogonal data which lends itself well to good performing embeddings based IR. Real-world data is mixed up, messy, and requires a lot of work to extract, clean, normalize, etc. Typically fine-tuned embedding model would be needed.
> Real-world data is mixed up, messy, and requires a lot of work to extract, clean, normalize, etc. Typically fine-tuned embedding model would be needed.
I'd think the former is required for whichever strategy you choose, although I haven't found it necessary to fine-tune a base LLM.
What use case are you thinking of when you say, "typically fine-tuning is required"?
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#9For anyone interested in a similar project supporting a range of data sources, check out www.aibrainbank.com
You create 'contexts' (like folders) of data to chat with, and your data can be any of the following:
- Text files, pdf files, etc - Audio files (automatic transcription) - Web pages - YouTube videos - Entire YouTube channels
Demo video on front page too.
Re: Build a chatbot with custom data sources, powered by LlamaIndex
#10I'd love to set up one of those fancy chatbots for my almost dead IRC channel. Does anyone know something out there?
We have something similar running today. Which acts as a bot for both Discord and an IRC channel.
Not sure tho if this code is shareable at the moment tho.
But if I'm not mistaken (can double check later on) it's based on Sopel
Ref: https://sopel.chat/