Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 Turbo fine-tuning and API updates

openai.com

Post reply on HN
Print view View original

81–90 of 244 posts

Re: GPT-3.5 Turbo fine-tuning and API updates

#81

Post by brianjking »

josephg wrote: post #75
brianjking wrote: post #47

This is honestly pretty disappointing. * No function calling support yet * Only 4k tokens, so can't use the full 16k token length. I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well.

> I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well. Literally the first sentence in the article: > Fine-tuning for GPT-3.5 Turbo is now available, with fine-tuning for GPT-4 coming this fall.

Yes, nowhere does it say anything about MULTIMODAL fine tuning being supported/unsupported when GPT-4 fine tuning becomes available.

This was my entire point. I did read the article.

Re: GPT-3.5 Turbo fine-tuning and API updates

#82

Post by brianjking »

htrp wrote: post #63
brianjking wrote: post #47

This is honestly pretty disappointing. * No function calling support yet * Only 4k tokens, so can't use the full 16k token length. I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well.

> multimodality of GPT-4 Is there a multimodal GPT 4 model in the wild? All I saw was that one example at launch.

Yes, just via Bing.com for now. The only people who have OpenAI's API access (officially) are the BeMyEyes app for vision impaired users.

InstructBlip is the SOTA model for open source otherwise.

Re: GPT-3.5 Turbo fine-tuning and API updates

#83

Post by 3abiton »

tuckerconnelly wrote: post #65

Question about privacy: OpenAI doesn't use API calls to train their models. But do they or Microsoft still store the text? If so, for how long? Overall, I think this is great, and can't wait for the 16k fine-tuning.

Does the finetuned model reside on OpenAI's servers? If so, what privacy guarantees that openai won't utilize it later for expanding gpt5?

Re: GPT-3.5 Turbo fine-tuning and API updates

#84

Post by notatoad »

simonw wrote: post #57
MPSimmons wrote: post #50

Earlier quoted context omitted.

I'm out of this particular loop. What does 'reduce prompt size' mean in this case? What is the advantage?

OpenAI models charge by the token (a token is usually about 3/4 of a word). A short prompt - "Say hi" - costs less money than a long prompt: "Five cute names for a pet weasel". You're charged for both the input and the output tokens: https://openai.com/pricing So for a fine-tuned model that costs 8x more to be a purely financial win, it would need to let you reduce the size of your input and output prompts by 8x or m…

that is actually hilarious. i've been enjoying ChatGPT instead of stack overflow precisely because it avoids that sort of judgemental answer - the AI just answers my question no matter how dumb it is, whereas stackoverflow will tell me the question is dumb whether it is or not. it's funny to see that llama has replicated stack overflow's judginess.

Re: GPT-3.5 Turbo fine-tuning and API updates

#85

Post by sebzim4500 »

swyx wrote: post #43
GaggiX wrote: post #15

Earlier quoted context omitted.

They would need to store and load the model, even if I imagine that they are using something similar to LoRA to finetune their models.

i would guess that the ideal price is also to raise the charge to make finetuning a last resort rather than a first resort; its probably much better cost- and research-wise if everybody just prompts the same model than silo off in their own minimodels.

I don't think I'd consider it a 'last resort', since a lot of people will be choosing between finetuned GPT-3.5 and non-finetuned GPT-4, in which case finetuning is the cheap option.

Re: GPT-3.5 Turbo fine-tuning and API updates

#87

Post by dinkleberg »

siva7 wrote: post #37
TheCaptain4815 wrote: post #9

Quite exciting, this is the first time their "Chat" model is available for finetuning. Has anyone successfully bypassed the current Ai detectors using fine-tuned models? I know it's possible, I'm just trying to conceptualize how the dataset would be organized...

ai detectors are scam products. it's not possible to detect with reliable certainty if text is human or chatgpt generated

In general I agree, though there are many cases where it is blatantly obvious something was written by AI. So many comments out there that are the classic ChatGPT 3-paragraphs where it has to restate the entire situation and then give you vague, non-committal answers.

Re: GPT-3.5 Turbo fine-tuning and API updates

#88

Post by flangola7 »

3abiton wrote: post #83
tuckerconnelly wrote: post #65

Question about privacy: OpenAI doesn't use API calls to train their models. But do they or Microsoft still store the text? If so, for how long? Overall, I think this is great, and can't wait for the 16k fine-tuning.

Does the finetuned model reside on OpenAI's servers? If so, what privacy guarantees that openai won't utilize it later for expanding gpt5?

Insist on such guarantees in the contact.

Re: GPT-3.5 Turbo fine-tuning and API updates

#90

Post by nunodonato »

lukev wrote: post #18

Can anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as mentioned in the docs) really influence the behavior of the base model that much, or was that just a toy example?

I did a lot of fine-tuning for an AI assistant I was building. I noticed a good improvement when I crossed the 200-300 samples (Q&A pairs)
Post reply on HN