I haven't explored the API yet, but their interface for GPT-4 has been getting increasingly worse over the past month.
Things that GPT-4 would easily, and correctly, reason through in April/May it just doesn't do any longer.
GPT-4 API General Availability
61–70 of 562 posts
Re: GPT-4 API General Availability
#62I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
Re: GPT-4 API General Availability
#63Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…
What’s the diff with 3.5turbo with instruct?
the existing 3.5turbo is what you would call a "chat" model.
The difference between them is that the chat models are much more... chatty - they're trained to act like they're in a conversation with you. The chat models generally say things "Sure, I can do that for you!", and "No problem! Here is". The conversation style is generally more inconsistent in it's style. It can be difficult to make it only return the result you want, and occasionally it'll keep talking anyway. It'll also talk in first person more, and a few things like that.
So if you're using it as an API for things like summarization, extracting the subject of a sentence, code editing, etc, then the chat model can be super annoying to work with.
Re: GPT-4 API General Availability
#64I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…
It's the continued alignment with fine-tuning that's degrading its responses.
You can apparently have it be nice or smart, but not both.
Re: GPT-4 API General Availability
#65I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
It’s cheaper to use OpenAI. If you have your own compute, sentence-transformers is just as good for most use cases.
Re: GPT-4 API General Availability
#66I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
Sure, but I don't know of any models you can get local access to that work nearly as well.
Re: GPT-4 API General Availability
#67I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
You would need to have a local copy of the GPT model, which are not exactly OpenAI's plans.
Re: GPT-4 API General Availability
#68I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…
I hit rate limits and “model is busy with other requests” frequently while just developing a highly concurrent agent app. Especially with the dated (e.g. -0613) or now -16k models.
Re: GPT-4 API General Availability
#69Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…
I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.
I'm assuming they will price it the same as normal gpt-3.5-turbo. I won't use it if it's more than 2x the price of turbo, because I can usually get turbo to do what I want, it just takes more tokens sometimes.
Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.
Re: GPT-4 API General Availability
#70I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
Yes. The best public embedding model is decent, but I expect it’s objectively worse than the best model from OpenAI.