Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 Turbo fine-tuning and API updates

openai.com

Post reply on HN
Print view View original

121–130 of 244 posts

Re: GPT-3.5 Turbo fine-tuning and API updates

#121

Post by sheepscreek »

msp26 wrote: post #93
minimaxir wrote: post #4

Generating from a finetuned GPT 3.5 Turbo is 8x the cost of generating from the base model, so you really have to be in the “reduce prompt size by 90%” bucket they mention to get cost effectiveness out of it.

8x is a big gap. I feel that it might be better to just use few-shot prompting and supply some examples (not applicable to every use case ofc) every prompt to steer the output. But it's worth experimenting with at least. Edit: no function calling (until later this fall) too. That's most of my usage right now so I'll pass. Curious on what the verdict on the OAI finetunes are. I remember reading this thread which indic…

There’s a place for this. Like fine tuning the model on advanced particle physics so that it can decipher cutting edge research papers more accurately, without teaching the model particle physics every time in the prompt.

Re: GPT-3.5 Turbo fine-tuning and API updates

#122

Post by J_Shelby_J »

stavros wrote: post #60
danielbln wrote: post #27

Earlier quoted context omitted.

Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.

Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.

Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module.

RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search.

I think the thing most people will struggle with is prepping the documents. Chunking them to the best token count for your use case for example.

https://github.com/shelby-as-a/shelby-as-a-service

Re: GPT-3.5 Turbo fine-tuning and API updates

#123

Post by thenickdude »

flangola7 wrote: post #89
Femolo wrote: post #36

We are closer and closer to a post DM society. ADR, supportdocs will be king. And we are finally seeing a new area of real knowledge work. Soon it will be easier to train ai than new people.

DM?

Post dungeon-master. AIs will take over the hosting of Dungeons and Dragons sessions.

Re: GPT-3.5 Turbo fine-tuning and API updates

#124

Post by callalex »

Havoc wrote: post #104
htrp wrote: post #62

Earlier quoted context omitted.

This one seems to be a deal-breaker, if you already know what types of language you want, why would you want openai moderating your parameter tuning set.

Why do you care at all, let alone "dealbreaker". You need a model specifically fine tuned towards something dangerous?

Dangerous to who/what? Moderating is always opinion based.

Re: GPT-3.5 Turbo fine-tuning and API updates

#125

Post by stavros »

J_Shelby_J wrote: post #122
stavros wrote: post #60

Earlier quoted context omitted.

Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.

Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module. RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search. I think the thin…

That's very helpful, thanks!

Re: GPT-3.5 Turbo fine-tuning and API updates

#127

Post by zarzavat »

jakeduth wrote: post #99
3abiton wrote: post #83

Earlier quoted context omitted.

Does the finetuned model reside on OpenAI's servers? If so, what privacy guarantees that openai won't utilize it later for expanding gpt5?

Yes they are stored on OpenAI's servers. The API calls are not used for model training per the TOS. However, not that I'm accusing OpenAI of anything, but there's no way to independently validate this. But their guarantee is clear for the API (the ChatGPT web app is different, but you can disable training if you give up the history feature). > At OpenAI, protecting user data is fundamental to our mission. We do not t…

It’s in principle possible to detect if a model has been trained on private data, e.g. if it can recite random data such as UUIDs that are not public. So if OpenAI were to break that promise, someone would notice and make it public. This is enough of a disincentive that I trust OpenAI will not do it.

Re: GPT-3.5 Turbo fine-tuning and API updates

#128

Post by tedsanders »

tuckerconnelly wrote: post #65

Question about privacy: OpenAI doesn't use API calls to train their models. But do they or Microsoft still store the text? If so, for how long? Overall, I think this is great, and can't wait for the 16k fine-tuning.

30 days maximum, in most cases: https://platform.openai.com/docs/models/default-usage-polici...

We don’t do anything sneaky with the stored data; literally the only purpose is to be able to investigate possible trust and safety violations for a brief period after they occur.

Re: GPT-3.5 Turbo fine-tuning and API updates

#129

Post by steno132 »

And OpenAI continues to ship.

As promised, they released GPT3.5 fine-tuning today. They opened GPT4 API access a few months ago. In a few months, they'll release GPT4 fine-tuning.

Many favor open source AI, and criticize OpenAI for not being open enough. But the most important thing is, OpenAI innovates. Fast.

Llama, Bard, FB's open source stuff is good but it's lightyears behind OpenAI. You have to credit them for that.

Re: GPT-3.5 Turbo fine-tuning and API updates

#130

Post by grubbyboy »

menzoic wrote: post #115
bennyschmidt wrote: post #114

GPT 3.5 is so bad it's useless to me - for writing it's too repetitive of the same kind of jargon, for coding it's wrong way too often. The NLP is also worse, I have to be more explicit. It's just an average chat bot IMO. GPT 4 @ $20/mo. is significantly better at everything, I use it for doing stuff in Angular lol - when you have an AI explaining the why behind everything, this over-engineered mess of a framework st…

Use code interpreter to upload your files and prompt it to ask you a serires of questions to know what to do next

Code interpreter is quite good. I used it to create graphs, convert csv to JSON, write a complex Bash script, and regex. It's impressive.
Post reply on HN