Cool use case, glad to see txtai [1] is helping (I'm the main dev for txtai). Since you're using txtai, this article I just wrote yesterday might be helpful: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai Looks like you've received a lot of great ideas here already though! 1 - https://github.com/neuml/txtai
Is anyone using self hosted LLM day to day and training it like a new employee
71–75 of 75 posts
Re: Is anyone using self hosted LLM day to day and training it like a new employee
#72Re: Is anyone using self hosted LLM day to day and training it like a new employee
#73Cool use case, glad to see txtai [1] is helping (I'm the main dev for txtai). Since you're using txtai, this article I just wrote yesterday might be helpful: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai Looks like you've received a lot of great ideas here already though! 1 - https://github.com/neuml/txtai
Thanks. I build something similar (didn't know it was called RAG) about 6 months ago. I found the most difficult part was interfacing with the existing systems and extracting the documentation out of those. Google docs, Notion, Slack. Do you have any advice on easier ways to do this? Are there any libraries around that make this task a bit lighter?
Re: Is anyone using self hosted LLM day to day and training it like a new employee
#74Earlier quoted context omitted.
he is the human that does this.
Right, but developers are significantly more expensive than customer support agents. Especially if the answers are already documented, and the rep just needs to be friendly in pointing customers to the right documentation. If you can free up 5 hours of week of dev time by spending $100/week on a support rep, that's a great rate for dev time. This assumes you're running a profitable business in the first place. If you…
Re: Is anyone using self hosted LLM day to day and training it like a new employee
#75Earlier quoted context omitted.
Right, but developers are significantly more expensive than customer support agents. Especially if the answers are already documented, and the rep just needs to be friendly in pointing customers to the right documentation. If you can free up 5 hours of week of dev time by spending $100/week on a support rep, that's a great rate for dev time. This assumes you're running a profitable business in the first place. If you…
What do you think of the quality of support provided by someone who spends 5 hours a week answering questions vs the primary developer? This isn't as straightforward as you think. If he can free up 5 hours a week for $500 and loses 10 customers because they didn't like the support from someone who spends 5 hours a week on this product they've never used, do you think he wins or loses?
In my experience, the quality of support is great!
A year into my business, I brought someone in to do customer support for five hours per week, and it freed up a huge amount of time and mental bandwidth.[0]
Since then, I've expanded support to two non-technical customer service reps and two support engineers. It's rare that I answer a question now because my team can generally answer anything I can.
>If he can free up 5 hours a week for $500 and loses 10 customers because they didn't like the support from someone who spends 5 hours a week on this product they've never used, do you think he wins or loses?
I think this is wildly unlikely in practice.
When you bring on a support rep, you don't just say, "Okay, you're on your own. Do the best that you can." You work to onboard them and make sure they're comfortable answering questions and asking for help when they need it. You also course-correct or jump in if they give someone an incorrect or unhelpful answer.
In my two years of having a support team, I can't think of a single time that a customer said they were dropping us because they were dissatisfied with our support.
Instead, what I've found is that the support team often does a better job than I could have because they have more bandwidth to dig deeply into issues.[1] On about 10% of support requests, customers go out of their way to tell us how satisfied they are with the help we gave them.
Documenting things and onboarding teammates is time-consuming and difficult, but the commenter I was originally responding to said that they've already done that work. They described themselves as "a solo developer answering emails that basically point people to various guides and FAQs I’ve published." If they're already at that point, bringing in a support rep is a no-brainer.
In my experience, the majority of support requests I receive are nontechnical. I sell a physical product, so it's probably higher than a pure SaaS, but it's still about 60% of questions are about billing or they deleted their invoice by mistake or they need to see our tax forms for internal paperwork. That's all stuff that a support rep can handle just as well as I can even if they know nothing about my product.
[0] https://mtlynch.io/solo-developer-year-4/#good-leadership-me...