GPT-4 API General Availability • Hacker News Board

Re: GPT-4 API General Availability

#161

Post by km3r » Thu, Jul 06, 2023, 9:07 PM UTC

esperent wrote: post #140

athyuttamre wrote: post #111

Earlier quoted context omitted.

(I'm an engineer at OpenAI) Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case? https://github.com/openai/openai-cookbook/blob/main/examples... We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have t…

It's kind of incredible how fast OpenAI (now also known as ClosedAI) is going through the enshittification process. Even Facebook took around a decade to reach this level. OpenAI has an amazing core product, but in the span of six months: * Went from an amazing and inspiring open company that even put "Open" in their name to a fully locked up commercial beast. * Non-existent customers support and all kinds of borderl…

> Use of cheap human labor in developing countries through shady anonymous companies (look up the company Sama who pay Kenyan workers about $1.5 an hour).

What is wrong about injecting millions into developing nations?

The rest I agree with, although I don't think it was ever really 'open' so its not getting shitty, it always was. Thankfully, "there is no moat" and other LLMs will be open, just a few months behind OpenAI

Re: GPT-4 API General Availability

#162

Post by H8crilA » Thu, Jul 06, 2023, 9:08 PM UTC

superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

The $5-$10 is probably the reason why they're killing those endpoints.

Re: GPT-4 API General Availability

#163

Post by benjamoon » Thu, Jul 06, 2023, 9:08 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

You should apply and use OpenAI on azure. We’ve got close to 1m tokens per minute capacity across 3 instances and the latency is totally fine, like 800ms average (with big prompts). They’ve just got the new 0613 models as well (they seem to be about 2 weeks behind OpenAI). We’ve been in production for about 3 months, have some massive clients with a lot traffic and our gpt bill is way under £100 per month. This is all 3.5 turbo though, not 4 (but that’s available on application, but we don’t need it).

Re: GPT-4 API General Availability

#164

Post by samstave » Thu, Jul 06, 2023, 9:08 PM UTC

superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

Please ELI5 if I am mis-interpretating what you said:

*"They have just locked down access to a model which they basically realized was way more valuable than even they thought - and they are in the process of locking in all controls around exploiting the model for great justice?"*

Re: GPT-4 API General Availability

#165

Post by RC_ITR » Thu, Jul 06, 2023, 9:09 PM UTC

leodriesch wrote: post #113

furyofantares wrote: post #78

Not a lot of talk of Whisper being available here. From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said. But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate. Given this is how OpenAI themselves use it (say your whole thing before getting…

I'm interested in how the transformer based speech recognition from iOS 17 will perform compared to Whisper. I guess it will work more "real-time" like the current dictation on iOS/macOS, but I'm unsure as I am not on the beta right now.

My guess is the reason that apple invested so heavily in this [0] is because they are going to train a big transformer in their datacenter and apply it as an RNN on your phone.

Superficially, I think this will work very well, but slightly worse than whisper (with the advantage ofc being that its better at real-time transcription).

[0]https://machinelearning.apple.com/research/attention-free-tr...

Re: GPT-4 API General Availability

#166

Post by whalesalad » Thu, Jul 06, 2023, 9:10 PM UTC

seizethecheese wrote: post #73

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

It definitely got nerfed.

Re: GPT-4 API General Availability

#167

Post by pamelafox » Thu, Jul 06, 2023, 9:10 PM UTC

swyx wrote: post #122

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

> Could it be just my region that is affected? as far as I know OpenAI only has one region, that is out in Texas. even more hilariously, as far as I can tell, Azure OpenAI -also- only has one region.. cant imagine why

You can see region availability here for Azure OpenAI:

https://learn.microsoft.com/en-us/azure/cognitive-services/o...

It's definitely limited, but there's currently more than one region available.

(I happen to be working at the moment on a location-related fix to our most popular Azure OpenAI sample, https://github.com/Azure-Samples/azure-search-openai-demo )

Re: GPT-4 API General Availability

#168

Post by benjamoon » Thu, Jul 06, 2023, 9:12 PM UTC

swyx wrote: post #122

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

> Could it be just my region that is affected? as far as I know OpenAI only has one region, that is out in Texas. even more hilariously, as far as I can tell, Azure OpenAI -also- only has one region.. cant imagine why

Totally wrong, Azure has loads of regions. We’re using 3 in our app (UK, France and US East). It’s rapid.

Re: GPT-4 API General Availability

#169

Post by superkuh » Thu, Jul 06, 2023, 9:13 PM UTC

H8crilA wrote: post #162

superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

The $5-$10 is probably the reason why they're killing those endpoints.

I don't get it? text-davinci-003 is the most expensive model per token. It's just that running IRC bots isn't exactly high volume.

Re: GPT-4 API General Availability

#170

Post by w10-1 » Thu, Jul 06, 2023, 9:15 PM UTC

EnnioEvo wrote: post #10

minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

I have a startup of legal AI, the quality jump from GPT3.5 to GPT4 in this domain is straight mind-blowing, GPT3.5 in comparison is useless. But I see how in more conversational settings GPT3.5 can provide more appealing performance/price.

Legal writing is ideal training data: mostly formulaic, based on conventions and rules, well-formed and highly vetted, with much of the best in the public domain.

Medical writing is the opposite, with unstated premises, semi-random associations, and rarely a meaningful sentence.