Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 Turbo fine-tuning and API updates

openai.com

Post reply on HN
Print view View original

61–70 of 244 posts

Re: GPT-3.5 Turbo fine-tuning and API updates

#61

Post by rsiqueira »

minimaxir wrote: post #4

Generating from a finetuned GPT 3.5 Turbo is 8x the cost of generating from the base model, so you really have to be in the “reduce prompt size by 90%” bucket they mention to get cost effectiveness out of it.

This means that the cost would be more than US$ 1 for each 16 user interactions, using this fine-tuning pricing: 16 interactions * ($0.012*4 input + $0.016 output) = US$ 1.02

It's 8x more expensive, indeed. I'm comparing with my use case, the standard gpt-3.5 API, where my users consume 4k input tokens (due to context plus chat history) and almost 1k output tokens.

Re: GPT-3.5 Turbo fine-tuning and API updates

#62

Post by htrp »

GaggiX wrote: post #3

"fine-tuning training data is passed through our Moderation API and a GPT-4 powered moderation system", that sounds expensive, I imagine they would use GPT-4 only when the "safety" is ambiguous, training a model is less expensive than running GPT-4 inference by looking at the API price.

This one seems to be a deal-breaker, if you already know what types of language you want, why would you want openai moderating your parameter tuning set.

Re: GPT-3.5 Turbo fine-tuning and API updates

#63

Post by htrp »

brianjking wrote: post #47

This is honestly pretty disappointing. * No function calling support yet * Only 4k tokens, so can't use the full 16k token length. I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well.

> multimodality of GPT-4

Is there a multimodal GPT 4 model in the wild? All I saw was that one example at launch.

Re: GPT-3.5 Turbo fine-tuning and API updates

#64

Post by josephg »

minimaxir wrote: post #40
alooPotato wrote: post #35

Earlier quoted context omitted.

I think you have to compare it to your alternative which is prob gpt4 in which case its much cheaper?

Correct, GPT-4 is 20-30x the cost compared to GPT-3.5-turbo, but that's more in another class entirely. IMO a finetuned-GPT-3.5-turbo won't beat base GPT-4 for the use cases where you really really need GPT-4, but that remains to be seen. The real alternative to a finetuned GPT-3.5-turbo is still the base model with a very very complicated system prompt.

From the article:

> Early tests have shown a fine-tuned version of GPT-3.5 Turbo can match, or even outperform, base GPT-4-level capabilities on certain narrow tasks.

It sounds like it really depends on what you're doing.

Re: GPT-3.5 Turbo fine-tuning and API updates

#66

Post by lukev »

stavros wrote: post #60
danielbln wrote: post #27

Earlier quoted context omitted.

Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.

Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.

https://www.trychroma.com

It works as a library, you can require the PIP package and get started immediately.

Re: GPT-3.5 Turbo fine-tuning and API updates

#67

Post by Sohcahtoa82 »

TheCaptain4815 wrote: post #9

Quite exciting, this is the first time their "Chat" model is available for finetuning. Has anyone successfully bypassed the current Ai detectors using fine-tuned models? I know it's possible, I'm just trying to conceptualize how the dataset would be organized...

> Has anyone successfully bypassed the current Ai detectors using fine-tuned models?

Current AI detectors are pure garbage. Anybody paying for one is getting scammed. Anybody using one to actually make decisions is making a grave error.

It's a real shame that some schools are using AI detectors to detect students using ChatGPT to write essays for them, because there have been many cases where the detectors flag essays as being AI-generated that are clearly written by hand.

All it takes is half an hour of playing with ChatGPT and asking it to write essays to understand ChatGPT's writing style. Yeah, with some decent prompting, you can get it to write in other styles, but let's be honest, anybody using ChatGPT to do their homework isn't typically putting in the effort to make it not look like ChatGPT.

Re: GPT-3.5 Turbo fine-tuning and API updates

#68

Post by Karrot_Kream »

Given that they're offering davinci-002 for completion, I was wondering whether they went back to offering an "unsafe" model, but both davinci-002 and babbage-002 enter an infinite loop for any "unsafe" completion where text-davinci-003 and text-curie-001 is fine. Sigh, I guess OpenAI really does not want to offer unsafe models for usage.

Re: GPT-3.5 Turbo fine-tuning and API updates

#70

Post by stavros »

lukev wrote: post #66
stavros wrote: post #60

Earlier quoted context omitted.

Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.

https://www.trychroma.com It works as a library, you can require the PIP package and get started immediately.

This looks great, thank you!
Post reply on HN