How is this different from chatbase.co ? (not affiliated)
don't know what chatbase.co is but it seems to not be free. OpenChat is (?)
Run and create custom ChatGPT-like bots with OpenChat
41–50 of 63 posts
Re: Run and create custom ChatGPT-like bots with OpenChat
#42This feels similar to MagmaChat [1] which I open sourced about a month ago. Except mine is in Ruby on Rails. [1] https://magmachat.ai
I think you need a small video and much more clear and simple marketing material. For example, these don't mean anything for me: "Boxcars, CableReady, and StimulusReflex"
Re: Run and create custom ChatGPT-like bots with OpenChat
#43Disclaimer: I am curating LLM-tools on github [1] A few thoughts: * allow for custom endpoint URLs, this way people can use open source LLMs with a fake openAI API backend like basaran[2] or llama-api-server[3] * look into better embedding methods for info-retrieval like InstructorEmbeddings or Document Summary Index * Don't use a single embedding per content item, use multiple to increase retrieval quality 1 https:/…
Do you have a list somewhere, most impressive project you’ve seen?
Re: Run and create custom ChatGPT-like bots with OpenChat
#44This may be a naive question. But is there a way to embed a chat bot like this that only queries the data we feed it, and not the universe of other stuff in gpt? Like I don’t want people using the chatbot in our product to query a good strawberry shortcake recipe. We just want them to query about data we allow it to query which is native to our business. Is this feasible? Thx.
I think Simon Wilson who has a big blog posted often mentioned this is impossible
Re: Run and create custom ChatGPT-like bots with OpenChat
#45Earlier quoted context omitted.
For most applications, packaging all the data and submitting it to OpenAI won't be feasible due to the limited token window size. I think the most common design pattern nowadays goes like this: 1. Chunk all your data (e.g. per paragraph of content) 2. Generate an embedding for each chunk 3. Index embeddings in a vector database 4. When a query comes in, find chunks relevant to the query (based on embeddings similarit…
I've seen this described as the common approach and argued for it but with my limited knowledge I have difficulties countering the argument that it would be best to just finetune the model with your own data. I don't think it is as much the context window size because you would chunk your data anyways. I think the counter argument is either that finetuning is limited by the risk of overfitting and catastrophic forget…
In contrast, What’s the flow for training or fine tuning your own model
Re: Run and create custom ChatGPT-like bots with OpenChat
#46Earlier quoted context omitted.
I've seen this described as the common approach and argued for it but with my limited knowledge I have difficulties countering the argument that it would be best to just finetune the model with your own data. I don't think it is as much the context window size because you would chunk your data anyways. I think the counter argument is either that finetuning is limited by the risk of overfitting and catastrophic forget…
In contrast, What’s the flow for training or fine tuning your own model
Once I dug in to the fine-tuning APIs [1] I realized that the phrase "training the model on your docs" often doesn't make sense for the use case people are trying to solve. You provide hundreds of input examples and tell the model how it should complete those prompts. Fine-tuning has a lot of use cases, but "keeping the LLM generally grounded in the facts of my website" is not one of them.
[1] https://platform.openai.com/docs/guides/fine-tuning/prepare-...
Re: Run and create custom ChatGPT-like bots with OpenChat
#47Earlier quoted context omitted.
In contrast, What’s the flow for training or fine tuning your own model
Once I dug in to the fine-tuning APIs [1] I realized that the phrase "training the model on your docs" often doesn't make sense for the use case people are trying to solve. You provide hundreds of input examples and tell the model how it should complete those prompts. Fine-tuning has a lot of use cases, but "keeping the LLM generally grounded in the facts of my website" is not one of them. [1] https://platform.openai…
Fine-tuning has a lot of use cases, but "keeping the LLM generally grounded in the facts of my website" is not one of them.
Yes, that's what everyone says and it makes total sense to me. I'm looking for (technical, but not too technical) arguments why it is not possible. There I'm not so much interested in the "grounded in the facts of my website" point but more in the similar "take the data from my large private knowledge base into consideration" point.
In other words I don't want to restrict the knowledge the model has or the answers it gives. I want to add a considerable amount of my own knowledge. This seems not to be possible without training from scratch. The question is "Why?"
Re: Run and create custom ChatGPT-like bots with OpenChat
#48I went ahead and installed it in a proxmox container, was fairly easy on x64 (arm support would be nice).
One suggestion: it would be nice to have a short-term memory - a la ChatGPT. With the token limit at 4-8k for GPT-4, it would be nice to take advantage of that with both the "long-term memory" (vector store) but also a "short-term" one (as in, sending the previous questions/answers for context).
Re: Run and create custom ChatGPT-like bots with OpenChat
#49This may be a naive question. But is there a way to embed a chat bot like this that only queries the data we feed it, and not the universe of other stuff in gpt? Like I don’t want people using the chatbot in our product to query a good strawberry shortcake recipe. We just want them to query about data we allow it to query which is native to our business. Is this feasible? Thx.
This is somewhat possible. I've created a way to chat to our company's material publicly. We used a lot of prompt engineering and custom guardrails to achieve this. However, it severely limited the length of the conversation that a user can have.
Re: Run and create custom ChatGPT-like bots with OpenChat
#50This may be a naive question. But is there a way to embed a chat bot like this that only queries the data we feed it, and not the universe of other stuff in gpt? Like I don’t want people using the chatbot in our product to query a good strawberry shortcake recipe. We just want them to query about data we allow it to query which is native to our business. Is this feasible? Thx.
When you're building applications on top of LLMs, there are a number of central problems that you're trying to solve and this is one of them. Solutions are numerous and widely variable, everything from basic regex parsing to fine-tuning validator models to new programming/modeling languages. Here's some examples:
- https://github.com/microsoft/guidance
- https://github.com/NVIDIA/NeMo-Guardrails/
- https://github.com/r2d4/rellm
- https://shreyar.github.io/guardrails/
- https://lmql.ai/
- https://github.com/jbrukh/gpt-jargon