[dead]
GPT-3.5 Turbo fine-tuning and API updates
231–240 of 244 posts
Re: GPT-3.5 Turbo fine-tuning and API updates
#232Could someone explain in simple terms exactly what fine-tuning does? Does it show the model how to answer questions, or does it give it new information, or both? Is there a way to restrict answers to the fine-tuned data? For example, if one would want to use an LLM to answer questions regarding a large, private knowledge base, would it make sense to fine-tune a model on this knowledge base? If yes, how does one reduc…
It's important to note that this fine-tuning is what is known as "supervised fine-tuning" where you give the LLM a set of question/answer pairs and it tunes to those (see https://huyenchip.com/2023/05/02/rlhf.html). This is quite different then fine-tuning the base model or doing RLHF (reinforcement learning from human feedback).
My guess is that it could work well to adjust the goal of the LLM, i.e. tell it to behave in a certain way, or do a different task than just being a generalist chat bot.
This is quite different than adding knowledge to the bot (known as grounded-generation or retrieval-augmented-generation), which aims to augment the base model with new data (e.g. your confidential data).
So in short - I think it's not appropriate for answering questions about a large private knowledge base and GG/RAG is better suited. (if you're interested, I wrote a blog article about this recently: https://vectara.com/fine-tuning-vs-grounded-generation/)
Re: GPT-3.5 Turbo fine-tuning and API updates
#233Earlier quoted context omitted.
I use it with a tool that is wired into my terminal that changes my files for me [1]. That alone makes me several times more productive compared to copy pasting back and forth between the chat window. If the chat window makes me twice as productive the command line tool probably makes me 5x as productive. At that kind of output on a developer salary the $70-200 a month is absolute peanuts compared to what you get in…
Is your code subject to code review? If so have you done anything to improve that bottleneck, or was it never an issue at previous productivity?
It is subject to code review, but I typically spent much more time writing code than having it reviewed (I am very methodical and slow writing code)
Re: GPT-3.5 Turbo fine-tuning and API updates
#234Earlier quoted context omitted.
I hope this technology keeps improving to the point we can run it on our own machines. It's too good to be censored.
> In one hour, the chatbots suggested four potential pandemic pathogens, explained how they can be generated from synthetic DNA using reverse genetics, supplied the names of DNA synthesis companies unlikely to screen orders, identified detailed protocols and how to troubleshoot them, and recommended that anyone lacking the skills to perform reverse genetics engage a core facility or contract research organization. ht…
I'm sorry. Are you implying I'm not supposed to know about any of those things you cited? That it's "sensitive" information, not meant for people like me?
Your post is the exact reason why we need uncensored models running in a distributed manner.
Re: GPT-3.5 Turbo fine-tuning and API updates
#235Earlier quoted context omitted.
There is a GTP-4 plugin available for this purpose. I have not tested it myself, but it may be worth trying out?
You are referring to this one, yes? https://recombinant.ai
That's seriously awesome. Does this require ChatGPT 4 subscription? I can't justify paying to work on open source.
Re: GPT-3.5 Turbo fine-tuning and API updates
#236Earlier quoted context omitted.
Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module. RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search. I think the thin…
I know I left another comment, but I'm replying in a new one to increase the chance that you'll see this: It would probably be very useful to a lot of people if you (or someone) made a standalone library out of this. It looks like Shelby requires Pinecone, but using something like ChromaDB for local embeddings and providing a library that I can just import, add documents to, and then query for answers would enable al…
That's actually something that I think is a next step.
a) run locally through a web ui b) run with a local vectorstore
From my perspective of someone using it as a bot that runs in a container using a cloud based vector store makes a lot of sense, but yeah that's really good feedback and not something I considered. Hrmmm. Ok let me get a look at some local vectorstores.
Re: GPT-3.5 Turbo fine-tuning and API updates
#237And OpenAI continues to ship. As promised, they released GPT3.5 fine-tuning today. They opened GPT4 API access a few months ago. In a few months, they'll release GPT4 fine-tuning. Many favor open source AI, and criticize OpenAI for not being open enough. But the most important thing is, OpenAI innovates. Fast. Llama, Bard, FB's open source stuff is good but it's lightyears behind OpenAI. You have to credit them for t…
>And OpenAI continues to ship. This. Google (specifically their CEO) was saying since at least 2016 that "Google is an AI first company". (Whatever the hell that means). But they had no product to show for and they are on the verge of being the next IBM.
> Google (specifically their CEO) was saying since at least 2016 that "Google is an AI first company". (Whatever the hell that means).
What it means is it's why so many things about Google experience suck so badly. Whether it's what he meant or not, the practical flip side of Google being "AI first company" is that they're "humans last" company. Or, it's a different way of saying they only do things that scale. Telemetry and automated decision making scale. Human review and customer support do not.
Re: GPT-3.5 Turbo fine-tuning and API updates
#238Earlier quoted context omitted.
It depends on your needs. For instance, do you want to host an API or do you want to have a front end like chatGPT? Chances are, text-generation-webui [1] should get you pretty close to hosting it yourself. You simply clone the repo, download the model from huggingface using the included helper (download-model.py) and fire up the server with server.py. You can connect to it by SSH port tunneling on port 7860 (there's…
If you want to query the Llama-2 models, you can use Anyscale Endpoints [1]. Note: I work on this :) Llama-2-70B is $1 / million tokens, which is the most cost-efficient on the market that I'm aware of. [1] https://app.endpoints.anyscale.com/
Tried to plug it in to my favorite chat frontend (TypingMind), bounced off CORS. Is this something you can do something about?
Re: GPT-3.5 Turbo fine-tuning and API updates
#239Could someone explain in simple terms exactly what fine-tuning does? Does it show the model how to answer questions, or does it give it new information, or both? Is there a way to restrict answers to the fine-tuned data? For example, if one would want to use an LLM to answer questions regarding a large, private knowledge base, would it make sense to fine-tune a model on this knowledge base? If yes, how does one reduc…
Andrew Ng's deeplearning.ai just released a course on this yesterday https://www.deeplearning.ai/short-courses/finetuning-large-l... - at a high level it lets one put more data into the model than fits into the prompt. And yes your use case of a large private knowledge base is one of the prime examples she used in the course. Scenarios that are domain-specific or privacy conscious probably makes more sense for finetu…
But can't you still achieve domain-specific or privacy conscious cases by using retrieval + vector dbs?
Re: GPT-3.5 Turbo fine-tuning and API updates
#240Earlier quoted context omitted.
Andrew Ng's deeplearning.ai just released a course on this yesterday https://www.deeplearning.ai/short-courses/finetuning-large-l... - at a high level it lets one put more data into the model than fits into the prompt. And yes your use case of a large private knowledge base is one of the prime examples she used in the course. Scenarios that are domain-specific or privacy conscious probably makes more sense for finetu…
But can't you still achieve domain-specific or privacy conscious cases by using retrieval + vector dbs?
Not an expert but my understanding is retrieval+vector db enables accurate information retrieval, whereas finetuning is more focused on specific task performance and generating coherent text (maybe in a different style) generally.