Any details on what changed in the -002 models?
GPT-3.5 Turbo fine-tuning and API updates
41–50 of 244 posts
Re: GPT-3.5 Turbo fine-tuning and API updates
#42Earlier quoted context omitted.
link for those who also wanted to look it up https://twitter.com/OfficialLoganK/status/169406294917713961...
That's a different link than the one I was thinking of, but I came here to post it all the same! Definitely encourage everyone to post in support of increased documentation and specific examples on why you'd use it.
which was your link?
Re: GPT-3.5 Turbo fine-tuning and API updates
#43Earlier quoted context omitted.
I wonder why is much more expensive.
They would need to store and load the model, even if I imagine that they are using something similar to LoRA to finetune their models.
i would guess that the ideal price is also to raise the charge to make finetuning a last resort rather than a first resort; its probably much better cost- and research-wise if everybody just prompts the same model than silo off in their own minimodels.
Re: GPT-3.5 Turbo fine-tuning and API updates
#44[deleted]
Re: GPT-3.5 Turbo fine-tuning and API updates
#45Generating from a finetuned GPT 3.5 Turbo is 8x the cost of generating from the base model, so you really have to be in the “reduce prompt size by 90%” bucket they mention to get cost effectiveness out of it.
From https://twitter.com/OfficialLoganK/status/169406294917713961...
Early testers have reduced prompt size by up to 90% by fine-tuning instructions into the model itself, speeding up each API call and cutting costs.
I wonder if that 90% is precisely due to the calculation you made.
Re: GPT-3.5 Turbo fine-tuning and API updates
#46Earlier quoted context omitted.
That's a different link than the one I was thinking of, but I came here to post it all the same! Definitely encourage everyone to post in support of increased documentation and specific examples on why you'd use it.
which was your link?
Just an older post about GPT-3.5 and fine tuning requests.
Re: GPT-3.5 Turbo fine-tuning and API updates
#47This is honestly pretty disappointing.
* No function calling support yet * Only 4k tokens, so can't use the full 16k token length.
I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well.
Re: GPT-3.5 Turbo fine-tuning and API updates
#48Are there major advantages of GPT-3.5 Turbo tuning over PEFT/LoRA with Llama2?
Re: GPT-3.5 Turbo fine-tuning and API updates
#49Earlier quoted context omitted.
Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.
There is a risk of (greater) hallucination if there aren't explicit sample inputs provided by RAG.
Finetuning teaches skills. RAG provides 'facts.'
Re: GPT-3.5 Turbo fine-tuning and API updates
#50Generating from a finetuned GPT 3.5 Turbo is 8x the cost of generating from the base model, so you really have to be in the “reduce prompt size by 90%” bucket they mention to get cost effectiveness out of it.
I'm out of this particular loop. What does 'reduce prompt size' mean in this case? What is the advantage?