251–260 of 562 posts
Earlier quoted context omitted.
From what I can find, pricing of GPT-4 is roughly 25x that of 3.5 turbo. https://openai.com/pricing https://platform.openai.com/docs/deprecations/
In this thread we’re talking about gpt-3.5-turbo-instruct, not GPT4
>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge…
Earlier quoted context omitted.
https://github.com/ggerganov/llama.cpp https://huggingface.co/TheBloke There's a LocalLLaMA subreddit, irc channels, and a whole big community around the web working on it on GitHub nd elsewhere. edit: I forgot to directly answer you: yes you can run these models. 16GB of plenty. Different quantizations give you different amounts of smarts and speed. There are tables that tell you how much RAM is needed per which qua…
A reminder that llama isn't legal for the vast majority of use cases. Unless you signed their contract and then you can use it only for research purposes.
All of these are surmountable problems.
We can beat OpenAI.
We can drain their moat.
Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.
Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options. I have an old laptop with 16GB RAM and no GPU. Can I run these models?
Here's a handy model comparison chart (this is a coding benchmark, so coding-only models tend to rank higher): https://i.imgur.com/AqSjjj2.jpeg
>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge…
This tells me that either there were very few commercial users of finetuned models, or they need to decommission the infrastructure to free up GPU's for more valuable projects.
I assume if you reach out they throw some credits at you
Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…
What's the difference between chat and instruction tuning?
Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.
> "Starting today, all paying API customers have access to GPT-4." OK maybe I'm stupid but I am a paying OpenAI API customer and I don't have it yet. I see: gpt-3.5-turbo-16k gpt-3.5-turbo gpt-3.5-turbo-16k-0613 gpt-3.5-turbo-0613 gpt-3.5-turbo-0301 I don't see any gpt-4 Edit: Probably my problem is that I upgraded to paid API account within the last month, so I'm not technically a "paying API customer" yet according…
> Today all existing API developers with a history of successful payments can access the GPT-4 API with 8K context. We plan to open up access to new developers by the end of this month, and then start raising rate-limits after that depending on compute availability. Same for me. I signed up only a few days ago and was excited to switch to "gpt-4" but I haven't paid the first bill (save the $5 capture) so I probably h…
Earlier quoted context omitted.
This tells me that either there were very few commercial users of finetuned models, or they need to decommission the infrastructure to free up GPU's for more valuable projects.
The former seems very believable. And I bet a lot of the fine tuned models that are active are still part of prototypes or experiments. I assume if you reach out they throw some credits at you
The extra trust gained by seeing another customer treated that way easily pays for a few credits for a small number of users.
Earlier quoted context omitted.
Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options. I have an old laptop with 16GB RAM and no GPU. Can I run these models?
16GB of RAM can fit a 5 bit 13B model at best, they're second dumbest class of LLama model. If Open Orca turns out any good than that might be enough for the time being, but you'll need more RAM to use anything serious. Here's a handy model comparison chart (this is a coding benchmark, so coding-only models tend to rank higher): https://i.imgur.com/AqSjjj2.jpeg
It beats Claude and Bard.
You could probably get a 4bit 15B model going in 16GB of RAM and be approaching GPT4 in capability.
...on an old laptop, lol
Let's eat OpenAI's lunch! They deserve it for trying to steal this tech by "privatizing" a charity, hiding scientific data that was supposed to be shared with us by said charity whose purpose was to help us all, and dishonestly trying to persuade the government not to let us compete with them.
