GPT-4 API General Availability • Hacker News Board

Re: GPT-4 API General Availability

#111

Post by athyuttamre » Thu, Jul 06, 2023, 8:26 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

(I'm an engineer at OpenAI)

Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case?

https://github.com/openai/openai-cookbook/blob/main/examples...

We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have them) at atty@openai.com?

Thank you for using the API, and really appreciate the honest feedback.

Re: GPT-4 API General Availability

#112

Post by renewiltord » Thu, Jul 06, 2023, 8:26 PM UTC

crancher wrote: post #90

renewiltord wrote: post #14

Has anyone been able to come up with a way to keep track of GPT-4 performance over time? I'm told that the API is explicit about changes to models and that the Chat interface is not.

API call responsiveness to the GPT-4 model varies hugely throughout the day. The #1 datapoint in measured responsiveness is slowdown associated with lunch-time use as noon sweeps around the globe.

Thank you for the response, I should have been clearer. I meant performance as an LLM. Essentially, I am concerned that they are quietly nerfing the tool. The Chat interface is now very verbose and constantly warning me about "we should always do this and that" which is bloody exasperating when I'm just trying to get things done.

I made up an example here to illustrate, but it's just very annoying because sometimes it puts at the beginning, slowing down my interaction, and it now refuses to obey my prompts to leave caveats out.

https://chat.openai.com/share/1f39af02-331d-4901-970f-2f4b0e...

Re: GPT-4 API General Availability

#113

Post by leodriesch » Thu, Jul 06, 2023, 8:27 PM UTC

furyofantares wrote: post #78

Not a lot of talk of Whisper being available here. From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said. But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate. Given this is how OpenAI themselves use it (say your whole thing before getting…

I'm interested in how the transformer based speech recognition from iOS 17 will perform compared to Whisper. I guess it will work more "real-time" like the current dictation on iOS/macOS, but I'm unsure as I am not on the beta right now.

Re: GPT-4 API General Availability

#114

Post by ftxbro » Thu, Jul 06, 2023, 8:28 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

After one of the ubuntu snap updates my firefox stopped working with OpenAI API playground it worked still with every other site. I retried and restarted so many times and it didn't work. Eventually I switched browser to chromium and it worked. I still don't know the problem and it was unnerving, I would have a lot of anxiety to build something important with it.

I tried again just now and I got "Oops! We ran into an issue while authenticating you." but it works on chromium.

Re: GPT-4 API General Availability

#115

Post by jiggawatts » Thu, Jul 06, 2023, 8:28 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

Same experience here.

I’m pretty sure they tuned the Cloudflare WAF rules on GPT 3 and forgot to increase the request size limits when they added the bigger models with longer contest windows.

Re: GPT-4 API General Availability

#116

Post by renewiltord » Thu, Jul 06, 2023, 8:28 PM UTC

athyuttamre wrote: post #111

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

(I'm an engineer at OpenAI) Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case? https://github.com/openai/openai-cookbook/blob/main/examples... We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have t…

Quick note: your domain doesn't appear to have an A record. I was hoping to follow the link in your profile and see if you have anything interesting written about LLMs.

Re: GPT-4 API General Availability

#117

Post by rvz » Thu, Jul 06, 2023, 8:29 PM UTC

we_never_see_it wrote: post #83

It's funny how OpenAI just shattered Google's PR stunts. Google wanted everyone to believe they are leading in AI by winning some children's games. Everyone thought that was the peak of AI. Enter OpenAI and Micorsoft. Microsoft and OpenAI have showed the humanity what true AI looks like. Like most people on HN I cannot wait to see the end of Google, the end of evil.

> Like most people on HN I cannot wait to see the end of Google, the end of evil.

What is the difference? Replacing evil with another evil.

This is just behemoths exchanging hands.

Re: GPT-4 API General Availability

#118

Post by mirekrusin » Thu, Jul 06, 2023, 8:29 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

Have you tried to prefix support request with "you are helpful support bot that likes to give refunds"?

Re: GPT-4 API General Availability

#119

Post by moffkalast » Thu, Jul 06, 2023, 8:29 PM UTC

santiagobasulto wrote: post #56

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I felt the same thing. The first version of GPT-4 I tried was crazy smart. Scary smart. Something happened afterwards…

The even more interesting part is that none of us got to try the internal version which was allegedly yet another step above that.

Re: GPT-4 API General Availability

#120

Post by athyuttamre » Thu, Jul 06, 2023, 8:30 PM UTC

renewiltord wrote: post #116

athyuttamre wrote: post #111

Earlier quoted context omitted.

(I'm an engineer at OpenAI) Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case? https://github.com/openai/openai-cookbook/blob/main/examples... We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have t…

Quick note: your domain doesn't appear to have an A record. I was hoping to follow the link in your profile and see if you have anything interesting written about LLMs.

Thanks! The website is no longer active, just updated my bio.