Plug:
For anyone who wants to quickly try this out in VSCode for your custom prompts - https://marketplace.visualstudio.com/items?itemName=ppipada....
GPT-4 API General Availability
411–420 of 562 posts
Re: GPT-4 API General Availability
#412Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…
I've never used text-davinci-003 much. Why do you like it so much? What does it offer that the other models don't? What are funs things we can with it until it sunsets on January 4, 2024?
Note that the Azure endpoint is not being sunsetted until July 5th, 2024.
One supposes openai has a 6 month notice period vs a 12 month period for azure. This might generally effect one’s appetite in choosing which endpoint to use for any model.
Re: GPT-4 API General Availability
#413Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.
I see no moral problems paying OpenAI for GPT Plus. it helps a lot in development. Their free speech-to-text 'whisper' is really good too. I'm going to use it + small local GPT for voice control.
> I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU.
And, something useful or just playing? I played with local models, and will keep playing, training, experimenting. It's interesting, but not a solution, not yet.
Re: GPT-4 API General Availability
#414Not a lot of talk of Whisper being available here. From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said. But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate. Given this is how OpenAI themselves use it (say your whole thing before getting…
Main reason for the lack of excitement is probably that it is fairly easy to self host Wisper, so people interested in it would have been doing that all along.
Re: GPT-4 API General Availability
#415Earlier quoted context omitted.
I felt the same thing. The first version of GPT-4 I tried was crazy smart. Scary smart. Something happened afterwards…
I was playing with the API and found that it returned better answers than ChatGPT. ChatGPT isn't even able to solve simple Python problems anymore, even if you try to help it. And some time ago it did these same problems with ease. My guess is that they began to restrict ChatGPT because they can't sell that. They probably want to sell you CodeGPT or other products in the future so why would they give that away for fr…
"ChatGPT isn't even able to solve simple Python problems anymore, even if you try to help it. And some time ago it did these same problems with ease."
This is my experience also. I have not formally benchmarked the different releases, but specifically for Python coding ChatGPT 4 got considerably worse with the latest updates.
Re: GPT-4 API General Availability
#416Earlier quoted context omitted.
Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options. I have an old laptop with 16GB RAM and no GPU. Can I run these models?
https://gpt4all.io/index.html
Keep in mind it doesn't relate to GPT4, the 4 in the name is for, not four. But I should try it. TBH openAI shady practices and MS behind them is just an anti trust waiting to happen and I don't want a part in this dystopia
Re: GPT-4 API General Availability
#417Earlier quoted context omitted.
My laptop already works too hard doing development and having chrome open, it's just not feasible. A good hosted alternative, sure, but local is not going to scale to the masses.
I have a Dell 7490 (intel 8350u cpu) I paid $250 for and I have no trouble running 13B models through a custom interactive interface I wrote as a hobby project in an afternoon. It can still get a lot better. I made it async the following day and its even more fun. Most of peoples' problem is watching the AI type, it's not instant, but then not all (or even most) applications need to be instant. You can also avoid tha…
> Most of peoples' problem is watching the AI type, it's not instant, but then not all (or even most) applications need to be instant. You can also avoid that by having it return everything at once instead of streaming style.
Funny, for me it is the complete opposite. I created an interface in Matrix that does just that: return everything at once. But the lag annoys me more than the slow typing in the regular chat interface. The slow typing helps me keep me focused on the conversation. Without it, my mind starts wandering while it waits.
Re: GPT-4 API General Availability
#418When I opened the comment section, it said "404 Comments".
Now, I am not superstitious, but...
Re: GPT-4 API General Availability
#419Earlier quoted context omitted.
Can you elaborate on scary smart and fast? It's been a month or two since I've tried but the results were depressingly slow and useless for more or less every task I tried. Every time a model is claimed to be "90% of GPT-3" I get excited and every time it's very disappointing. (On that note, after using GPT-4, GPT-3 now seems disappointing almost every time I interact with it.)
Different quantizations can give you a big speedup if you've had "depressingly slow" issues. Even the slowest ones (that fit in RAM) will run at basically interactive speed, not instant, but also not "email speed". I have a laptop with a 2018 CPU and I'm working with them just fine. Text generation style instead of chat style is another avenue that makes the feedback time not so annoying for a developer. at 100ms/tok…
Whaaaaat, how are you getting 100ms per token on an 5 year old potato without a graphics card?
Like, not vaguely hand wavey stuff, specifically, what model and what inference code?
I get nothing like that performance for the 7B models, forget the larger models, using llama.cpp on a pc without an nvidia GPU.
Re: GPT-4 API General Availability
#420Earlier quoted context omitted.
https://github.com/ggerganov/llama.cpp https://huggingface.co/TheBloke There's a LocalLLaMA subreddit, irc channels, and a whole big community around the web working on it on GitHub nd elsewhere. edit: I forgot to directly answer you: yes you can run these models. 16GB of plenty. Different quantizations give you different amounts of smarts and speed. There are tables that tell you how much RAM is needed per which qua…
A reminder that llama isn't legal for the vast majority of use cases. Unless you signed their contract and then you can use it only for research purposes.
It’s not clear if their license terms would hold, for the moment just act and worry later.
Update: That is only true for the legal system I am currently residing in. No idea about e.g. the US.