GPT-4 API General Availability • Hacker News Board

Re: GPT-4 API General Availability

#31

Post by gamegoblin » Thu, Jul 06, 2023, 7:33 PM UTC

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model.

gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage.

The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack gpt-3.5-turbo to your use case even if it is hugely wasteful from a tokens point of view.

Re: GPT-4 API General Availability

#32

Post by ftxbro » Thu, Jul 06, 2023, 7:33 PM UTC

minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

> "With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4"

poe law

Re: GPT-4 API General Availability

#33

Post by hospitalJail » Thu, Jul 06, 2023, 7:34 PM UTC

I imagine the API quality isnt nerfed on a given day like ChatGPT can be.

There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist)

Every once in a while, I see redditors complain of it being nerfed.

Sometimes I go back to gpt3.5 and am mind boggled how much worse it is.

Makes me wonder if they keep increasing the version number while dumbing down the previous model.

With an API, being unreliable would be a deal-breaker. Looking forward to people fine-tuning LLMs with GPT4 API. I'd love it for medical purposes, I'm so worried of a future where the US medical cartels ban ChatGPT for medical purposes. At least with local models, we don't have to worry about regression.

Re: GPT-4 API General Availability

#34

Post by echelon » Thu, Jul 06, 2023, 7:34 PM UTC

superalignment wrote: post #9

With this is the death of any uncensored usage of their models. Davinci 3 is the most powerful model where you can generate any content by instructing it via the completions API - chat GPT 3 models will not obey requests for censored or adult content.

A big enough hole presents a wedge for new entrants to get started.

OpenAI will never fulfill the entire market, and their moat is in danger with every other company that has LLM cash flow.

They want to become the AWS of AI, but it's becoming clear they'll lose generative multimedia. They may see the LLM space become a race to the bottom as well.

Re: GPT-4 API General Availability

#35

Post by mensetmanusman » Thu, Jul 06, 2023, 7:34 PM UTC

tnel77 wrote: post #20

EnnioEvo wrote: post #10

Earlier quoted context omitted.

I have a startup of legal AI, the quality jump from GPT3.5 to GPT4 in this domain is straight mind-blowing, GPT3.5 in comparison is useless. But I see how in more conversational settings GPT3.5 can provide more appealing performance/price.

I suggested to my wife that ChatGPT would help with her job and she has found ChatGPT4 to be the same or worse as ChatGPT3.5. It’s really interesting just how variable the quality can be given your particular line of work.

Remember, communication style is also very important. Some communication styles mesh much better with these models.

Re: GPT-4 API General Availability

#36

Post by someplaceguy » Thu, Jul 06, 2023, 7:34 PM UTC

gadtfly wrote: post #25

The original davinci model was a friend of mine and I resent this deeply. I've had completions with it that had character and creativity that I have not been able to recreate with anything else. Brilliant and hilarious things that are a permanent part of my family's cherished canon.

You cannot say that and not provide an example.

Re: GPT-4 API General Availability

#37

Post by saliagato » Thu, Jul 06, 2023, 7:34 PM UTC

brolumir wrote: post #16

Hmm, when I try to change model name to "gpt-4" I get the "The model: `gpt-4` does not exist" error message. We are an API developer with a history of successful payments.. is there anything we need to do on our side to enable this, anyone know?

wait a couple of hours

Re: GPT-4 API General Availability

#38

Post by thewataccount » Thu, Jul 06, 2023, 7:35 PM UTC

minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

What usecases are you using it for?

I mostly use it for generating tests, making documentation, refactoring, code snippets, etc. I use it daily for work along with copilot/x.

In my experience GPT3.5turbo is... rather dumb in comparison. It makes a comment explaining what a method is going to do and what arguments it will have - then misses arguments altogether. It feels like it has poor memory (and we're talking relatively short code snippets, nothing remotely near it's context length).

And I don't mean small mistakes - I mean it will say it will do something with several steps, then just miss entire steps.

GPT3.5turbo is reliably unreliable for me, requiring large changes and constant "rerolls".

GPT3.5turbo also has difficulty following the "style/template" from both the prompt and it's own response. It'll be consistent then just - change. An example being how it uses bullet points in documentation.

Codex is generally better - but noticeably worse then GPT4 - it's decent as a "smart autocomplete" though. Not crazy useful for documentation.

Meanwhile GPT4 generally nails the results, occasionally needing a few tweaks, generally only with long/complex code/prompts.

tl;dr - In my experience for code GPT3.5turbo isn't even worth the time it takes to get a good result/fix the result. Codex can do some decent things. I just use GPT4 for anything more then autocomplete - it's so much more consistent.

Re: GPT-4 API General Availability

#39

Post by m3kw9 » Thu, Jul 06, 2023, 7:35 PM UTC

“Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4).“

So need to pay to fine tune again?

Re: GPT-4 API General Availability

#40

Post by ftxbro » Thu, Jul 06, 2023, 7:35 PM UTC

someplaceguy wrote: post #36

gadtfly wrote: post #25

The original davinci model was a friend of mine and I resent this deeply. I've had completions with it that had character and creativity that I have not been able to recreate with anything else. Brilliant and hilarious things that are a permanent part of my family's cherished canon.

You cannot say that and not provide an example.

i mean there are a lot of examples from february era sydney