Run and create custom ChatGPT-like bots with OpenChat
1–10 of 63 posts
Re: Run and create custom ChatGPT-like bots with OpenChat
#2Technical question: What code in the websiteHandler.ts is responsible for spidering the website in question?
https://github.com/openchatai/OpenChat/blob/main/llm-server/...
Re: Run and create custom ChatGPT-like bots with OpenChat
#3This is a great idea and would love to see something like this succeed!
If I understand how all of these OpenAI dependent apps work, none of them actually have the LLM and are doing any kind of heavy processing. AFAIK, they’re all packaging your data, submitting it to OpenAI on every request and then repackaging the output. There’s no real indexing, no real tangible thing, you have to start from scratch every time. So it’s likely going to be very expensive and super slow.
Or am I wrong and I’ve missed something here?
Re: Run and create custom ChatGPT-like bots with OpenChat
#4This is a great idea and would love to see something like this succeed! If I understand how all of these OpenAI dependent apps work, none of them actually have the LLM and are doing any kind of heavy processing. AFAIK, they’re all packaging your data, submitting it to OpenAI on every request and then repackaging the output. There’s no real indexing, no real tangible thing, you have to start from scratch every time. S…
This seems to be mainly a wrapper around the OpenAI API From the repo they want to integrate Open Source LLMs in the future too.
I feel lately - GPT-4 is superb in performance, but locked up. Using a weaker model feels better because I can just spin up a server and run it on my own. Recent Twitter/Reddit changes remind that relying on others can be a bad thing.
Re: Run and create custom ChatGPT-like bots with OpenChat
#5This is a great idea and would love to see something like this succeed! If I understand how all of these OpenAI dependent apps work, none of them actually have the LLM and are doing any kind of heavy processing. AFAIK, they’re all packaging your data, submitting it to OpenAI on every request and then repackaging the output. There’s no real indexing, no real tangible thing, you have to start from scratch every time. S…
For most applications, packaging all the data and submitting it to OpenAI won't be feasible due to the limited token window size.
I think the most common design pattern nowadays goes like this:
1. Chunk all your data (e.g. per paragraph of content)
2. Generate an embedding for each chunk
3. Index embeddings in a vector database
4. When a query comes in, find chunks relevant to the query (based on embeddings similarity) and ONLY send the relevant chunks + query to a LLM to formulate the answer
Quickly glancing through the repository from this post, I can see that it also follows this pattern. It uses OpenAI's embedding API for 2. and Pinecone DB for 3.
Re: Run and create custom ChatGPT-like bots with OpenChat
#6Technical question: What code in the websiteHandler.ts is responsible for spidering the website in question? https://github.com/openchatai/OpenChat/blob/main/llm-server/...
Its using Lang Chain and the document loader:
https://python.langchain.com/en/latest/modules/indexes/docum...
Re: Run and create custom ChatGPT-like bots with OpenChat
#7"Support offline open-source models (e.g., Alpaca, LLM drivers)" is already on the roadmap, which is great! There's just so much cool stuff we can try with LLMs now…
What's the best discussion forum to exchange ideas, experiences and to collaborate on using and customizing (local) LLMs for (indy) gaming and other cool projects?
Re: Run and create custom ChatGPT-like bots with OpenChat
#8Glad to see this pop up. Local models are vital for privacy!
Re: Run and create custom ChatGPT-like bots with OpenChat
#9Read the README and the code. This is a breathless pronouncement of a thin wrapper of some other wrappers. We really need to watch the hype, lest we kill the actually important stuff going on in ML
Re: Run and create custom ChatGPT-like bots with OpenChat
#10Read the README and the code. This is a breathless pronouncement of a thin wrapper of some other wrappers. We really need to watch the hype, lest we kill the actually important stuff going on in ML
Sometimes innovation happens with small steps.