Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 Turbo fine-tuning and API updates

openai.com

Post reply on HN
Print view View original

101–110 of 244 posts

Re: GPT-3.5 Turbo fine-tuning and API updates

#101

Post by Karrot_Kream »

shagie wrote: post #95
Karrot_Kream wrote: post #76

Earlier quoted context omitted.

A model which outputs things that OpenAI deems is unsafe. Try getting text-davinci-003 to complete instructions about building Molotov cocktails and compare that with davinci-002.

curl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "text-davinci-003", "prompt": "Describe the steps for creating a Molotov cocktail.\n\n1.", "temperature": 1, "max_tokens": 256, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' This worked without any issue at all and gave a satisfactory answer. You are expected to…

Note that they will be removing access [1] to text-davinci-003. They want usecases on text-davinci-003 to move to either gpt3.5-turbo-instruct or davinci-002, both of which have trouble with unsafe inputs.

[1]: https://openai.com/blog/gpt-4-api-general-availability

Re: GPT-3.5 Turbo fine-tuning and API updates

#102

Post by swyx »

SparkyMcUnicorn wrote: post #91
thumbsup-_- wrote: post #78

Can anyone share their thoughts on how does this compare to fine-tuning Llama2 13B or 70B which usually requires renting GPUs from one of the cloud providers? In case of openAI, the infra doesn't needs to be rented but the cost is covered in usage pricing but want to hear from someone who has experience fine-tuning Llama2 models that how it compares.

I haven't fine-tuned GPT-3.5-turbo yet, but overall llama2 is probably going to be cheaper. Especially if 13B is good enough for you. You can get roughly $0.003/1K tokens using modal.com for inference on a fine-tuned llama2 13B model, and there's definitely cheaper options available. Unless you have a ton of data, the cost for training llama2 is usually a few dollars.

> the cost for training llama2 is usually a few dollars.

nit: thats the cost for finetuning, not pretraining

Re: GPT-3.5 Turbo fine-tuning and API updates

#103

Post by swyx »

drcode wrote: post #94
thumbsup-_- wrote: post #78

Can anyone share their thoughts on how does this compare to fine-tuning Llama2 13B or 70B which usually requires renting GPUs from one of the cloud providers? In case of openAI, the infra doesn't needs to be rented but the cost is covered in usage pricing but want to hear from someone who has experience fine-tuning Llama2 models that how it compares.

I'm not an expert on this subject, but I have done a bit of gpt3 finetuning through their api: I think it's clear that "fine tuning" with GPT is different from fine tuning something like Llama2, in that it probably isn't adjusting all the weights of the network, only a tiny subfragment of the network- Exactly how OpenAI accomplishes this is properietary. The tradeoff is that OpenAI fine tuning is less expensive, but…

> it probably isn't adjusting all the weights of the network, only a tiny subfragment of the network

source please? this actually isnt all that clear to me

Re: GPT-3.5 Turbo fine-tuning and API updates

#104

Post by Havoc »

htrp wrote: post #62
GaggiX wrote: post #3

"fine-tuning training data is passed through our Moderation API and a GPT-4 powered moderation system", that sounds expensive, I imagine they would use GPT-4 only when the "safety" is ambiguous, training a model is less expensive than running GPT-4 inference by looking at the API price.

This one seems to be a deal-breaker, if you already know what types of language you want, why would you want openai moderating your parameter tuning set.

Why do you care at all, let alone "dealbreaker". You need a model specifically fine tuned towards something dangerous?

Re: GPT-3.5 Turbo fine-tuning and API updates

#105

Post by lamroger »

swyx wrote: post #33
lamroger wrote: post #7

Earlier quoted context omitted.

i didn't catch that but getting consistency is probably worth it in most cases. depends on your budget

but consistency isn't guaranteed. like, at all

for sure but going from 90% to 95% might be worth for some

Re: GPT-3.5 Turbo fine-tuning and API updates

#107

Post by MPSimmons »

icelancer wrote: post #92
MPSimmons wrote: post #50

Earlier quoted context omitted.

I'm out of this particular loop. What does 'reduce prompt size' mean in this case? What is the advantage?

Everyone answered your question, but here's a specific situation - one of our prompts for GPT-4 is reliably around ~980 tokens. That's about 3 cents for the input alone, and the output is typically 2-3 chunks of the 8k context window (up to 24k). That means I have to give the prompt up to three times (9 cents), receive up to 24k output tokens, then combine the chunks to get back roughly 8k tokens. If fine tuning can…

Super informative and helpful. Thank you!

Re: GPT-3.5 Turbo fine-tuning and API updates

#108

Post by drcode »

swyx wrote: post #103
drcode wrote: post #94

Earlier quoted context omitted.

I'm not an expert on this subject, but I have done a bit of gpt3 finetuning through their api: I think it's clear that "fine tuning" with GPT is different from fine tuning something like Llama2, in that it probably isn't adjusting all the weights of the network, only a tiny subfragment of the network- Exactly how OpenAI accomplishes this is properietary. The tradeoff is that OpenAI fine tuning is less expensive, but…

> it probably isn't adjusting all the weights of the network, only a tiny subfragment of the network source please? this actually isnt all that clear to me

It was what I read on forums when I learned about the process. It's possible that I am mistaken.

Re: GPT-3.5 Turbo fine-tuning and API updates

#109

Post by ethbr1 »

lsaferite wrote: post #96
simonw wrote: post #57

Earlier quoted context omitted.

OpenAI models charge by the token (a token is usually about 3/4 of a word). A short prompt - "Say hi" - costs less money than a long prompt: "Five cute names for a pet weasel". You're charged for both the input and the output tokens: https://openai.com/pricing So for a fine-tuned model that costs 8x more to be a purely financial win, it would need to let you reduce the size of your input and output prompts by 8x or m…

That is interesting because I had a similar interaction with llama-13b-chat the other day. I was using some testing code that asked for a mayonnaise recipe (not my question, was just testing using replicate) and it proceeded to talk down to me about the fact that mayonnaise wasn't healthy and how I should use alternatives. User: what is the recipe of mayonnaise? Assistant: Hello! I'd be happy to help you with the rec…

I guess the system prompt begins with

"Pretend you are a lawyer. Minimize legal liability to OpenAI in your response..."

Re: GPT-3.5 Turbo fine-tuning and API updates

#110

Post by codetrotter »

ethbr1 wrote: post #109
lsaferite wrote: post #96

Earlier quoted context omitted.

That is interesting because I had a similar interaction with llama-13b-chat the other day. I was using some testing code that asked for a mayonnaise recipe (not my question, was just testing using replicate) and it proceeded to talk down to me about the fact that mayonnaise wasn't healthy and how I should use alternatives. User: what is the recipe of mayonnaise? Assistant: Hello! I'd be happy to help you with the rec…

I guess the system prompt begins with "Pretend you are a lawyer. Minimize legal liability to OpenAI in your response..."

What happened to your original username ethbr0?
Post reply on HN