Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 Turbo fine-tuning and API updates

openai.com

Post reply on HN
Print view View original

171–180 of 244 posts

Re: GPT-3.5 Turbo fine-tuning and API updates

#171

Post by facia »

blorenz wrote: post #154
syntaxing wrote: post #149

Earlier quoted context omitted.

GPT-4 is significantly more expensive so I can definitely see you spending that amount. For really complex stuff, I switch over the GPT-4 and it will cost me almost $3 a "question" (as in going from the beginning to solving it). Honestly worth it since it solves my problem but it adds up quick so I try to stick with 3.5 when I can.

Can’t you get by with ChatGPT-4 for these personal assistant type questions? That’s what I do and my 20 a month goes a long way. I’d be interested to see if I am missing out on anything using GPT to is way in contrast to the API.

[dead]

Re: GPT-3.5 Turbo fine-tuning and API updates

#172

Post by SOLAR_FIELDS »

blorenz wrote: post #154
syntaxing wrote: post #149

Earlier quoted context omitted.

GPT-4 is significantly more expensive so I can definitely see you spending that amount. For really complex stuff, I switch over the GPT-4 and it will cost me almost $3 a "question" (as in going from the beginning to solving it). Honestly worth it since it solves my problem but it adds up quick so I try to stick with 3.5 when I can.

Can’t you get by with ChatGPT-4 for these personal assistant type questions? That’s what I do and my 20 a month goes a long way. I’d be interested to see if I am missing out on anything using GPT to is way in contrast to the API.

I use it with a tool that is wired into my terminal that changes my files for me [1]. That alone makes me several times more productive compared to copy pasting back and forth between the chat window. If the chat window makes me twice as productive the command line tool probably makes me 5x as productive. At that kind of output on a developer salary the $70-200 a month is absolute peanuts compared to what you get in return

1: https://github.com/paul-gauthier/aider

Re: GPT-3.5 Turbo fine-tuning and API updates

#173

Post by junofan »

speedgoose wrote: post #170
lynx23 wrote: post #164

Safety standards. One might call it "our censorship model" but that would be too close to the truth, so call it safety standards...

Do people really wish for OpenAI models to output healthcare advices without disclaimers, plans to end humanity, child porn fictions, torture instruction books, and similar? The minute it happens without complicated bypasses, the society would say stop to generative ai, and rightfully so. Many people already got spoked when they tricked ChatGPT to say/repeat scary things.

You can get all of that already by typing 4chan.org. Society seems unaffected.

Re: GPT-3.5 Turbo fine-tuning and API updates

#174

Post by robertnishihara »

bennyschmidt wrote: post #114

GPT 3.5 is so bad it's useless to me - for writing it's too repetitive of the same kind of jargon, for coding it's wrong way too often. The NLP is also worse, I have to be more explicit. It's just an average chat bot IMO. GPT 4 @ $20/mo. is significantly better at everything, I use it for doing stuff in Angular lol - when you have an AI explaining the why behind everything, this over-engineered mess of a framework st…

I think for fine-tuned GPT-3.5 to be competitive with GPT-4 on your use cases (assistance with Angular), you'd have to fine-tune on enough data that it really resembles pre-training more than fine-tuning. And it wouldn't be worth the hassle unless you're building a product around it.

That said, many valuable LLM products / features are more narrow in scope and can see a huge lift from fine-tuning. We've run a bunch of experiments on this (e.g., SQL query generation is a good example), where fine-tuning even the 7B Llama-2 model outperforms GPT-4 (surprisingly) [1]. That's a very different type of problem from teaching software engineering of course.

[1] https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...

Re: GPT-3.5 Turbo fine-tuning and API updates

#175

Post by manjoh »

syntaxing wrote: post #148
jimmcslim wrote: post #142

Earlier quoted context omitted.

Out of curiosity and if you are happy to share, what is your 'personal stuff'?

I use it a lot for personal coding projects, grammar correction/sentence rewording, and translation (it works better than google translate for longer text). I explicitly call out personal stuff since my job provides an in-house front end that uses the GPT API (I'm actually not sure which version it is, but guessing from the response quality, its probably GPT-4). My work one has made me noticeably more productive. It…

Can you talk about how you integrate gpt API at work and why not just use chatgpt 4?

Re: GPT-3.5 Turbo fine-tuning and API updates

#176

Post by ozr »

syntaxing wrote: post #133
steno132 wrote: post #129

And OpenAI continues to ship. As promised, they released GPT3.5 fine-tuning today. They opened GPT4 API access a few months ago. In a few months, they'll release GPT4 fine-tuning. Many favor open source AI, and criticize OpenAI for not being open enough. But the most important thing is, OpenAI innovates. Fast. Llama, Bard, FB's open source stuff is good but it's lightyears behind OpenAI. You have to credit them for t…

Prior to LLaMA 2, I would have agreed with you but LLaMA 2 is a game changer. The 70B performance is probably between 3.5 and 4. But running it personally isn't cheap. The cheapest I found is about $4/hr to run the whole thing. I only spend around $3 on average a month on GPT-3.5 API for my personal stuff.

I run 70B very cheaply using serverless GPUs. I've had the best experience with Runpod, but there are a few other options out there for it as well.

Re: GPT-3.5 Turbo fine-tuning and API updates

#177

Post by robertnishihara »

lukev wrote: post #18

Can anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as mentioned in the docs) really influence the behavior of the base model that much, or was that just a toy example?

We've run experiments on datasets ranging from 5K - 100K examples, which gave fantastic results [1].

Some examples - https://huggingface.co/datasets/b-mc2/sql-create-context - https://huggingface.co/datasets/GEM/viggo

On the other hand, 8K examples was not enough to learn to solve grade school math problems [2], so it is very problem dependent.

[1] https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...

[2] https://huggingface.co/datasets/gsm8k

Re: GPT-3.5 Turbo fine-tuning and API updates

#178

Post by andrewmunn »

robertnishihara wrote: post #168
syntaxing wrote: post #150

Earlier quoted context omitted.

It depends on your needs. For instance, do you want to host an API or do you want to have a front end like chatGPT? Chances are, text-generation-webui [1] should get you pretty close to hosting it yourself. You simply clone the repo, download the model from huggingface using the included helper (download-model.py) and fire up the server with server.py. You can connect to it by SSH port tunneling on port 7860 (there's…

If you want to query the Llama-2 models, you can use Anyscale Endpoints [1]. Note: I work on this :) Llama-2-70B is $1 / million tokens, which is the most cost-efficient on the market that I'm aware of. [1] https://app.endpoints.anyscale.com/

How do you keep the cost down?

Re: GPT-3.5 Turbo fine-tuning and API updates

#179

Post by Jach »

SOLAR_FIELDS wrote: post #172
blorenz wrote: post #154

Earlier quoted context omitted.

Can’t you get by with ChatGPT-4 for these personal assistant type questions? That’s what I do and my 20 a month goes a long way. I’d be interested to see if I am missing out on anything using GPT to is way in contrast to the API.

I use it with a tool that is wired into my terminal that changes my files for me [1]. That alone makes me several times more productive compared to copy pasting back and forth between the chat window. If the chat window makes me twice as productive the command line tool probably makes me 5x as productive. At that kind of output on a developer salary the $70-200 a month is absolute peanuts compared to what you get in…

Is your code subject to code review? If so have you done anything to improve that bottleneck, or was it never an issue at previous productivity?

Re: GPT-3.5 Turbo fine-tuning and API updates

#180

Post by greatpostman »

nacs wrote: post #166
greatpostman wrote: post #163

Earlier quoted context omitted.

I love how far behind Facebook and google are. I’ve always said they weren’t hiring the best, but rather are monopolies sucking the air out of the room. This proves it.

How is Facebook behind? They've released the most powerful open source LLM models so far (Llama 1 + 2) and are a serious threat to the Openai closed-source monopoly.

You can’t use llama2 like you can GPT4, it’s not even close.
Post reply on HN