Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

101–110 of 562 posts

Re: GPT-4 API General Availability

#101

Post by feoren »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

> you will get a 4xx/5xx response code, but you will still get billed for the request and whatever the servers generated and you didn't get. That's borderline fraudulent.

Borderline!? They're regularly charging customers for products they know weren't delivered. That sounds like straight-up fraud to me, no borderline about it.

Re: GPT-4 API General Availability

#102

Post by interstice »

merpnderp wrote: post #64
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

It's the continued alignment with fine-tuning that's degrading its responses. You can apparently have it be nice or smart, but not both.

Curious as to whether theres a more general rule at play there about filtering interfering with getting good answers. If there is that's a scary thought from an ethics perspective.

Re: GPT-4 API General Availability

#103

Post by sroussey »

hungrigekatze wrote: post #96
seizethecheese wrote: post #73

Earlier quoted context omitted.

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

Check out this post from a round table dialogue with Greg Brockman from OpenAI. The GPT models that were in existence / in use in early 2023 were not the performance-degraded quantized versions that are in production now: https://www.reddit.com/r/mlscaling/comments/146rgq2/chatgpt_...

Oh interesting. I thought that’s what turbo was.

Re: GPT-4 API General Availability

#104

Post by throwaway9274 »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

The click through API is mainly for prototyping.

If you want better latency and sane billing you need to go through Azure OpenAI Services.

OpenAI also offers decreased latency under the Enterprise Agreement.

Re: GPT-4 API General Availability

#105

Post by refulgentis »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

I understand your general point and am sympathetic to it, if you're a 10/10 on some scale, I'm about a 3-4. I've never seen billings for failures, but the billing stuff is crazy: no stats if you do streamed chat, and the only tokenizer available is in Python and for GPT-3.0.

However, I'm virtually certain somethings wrong on your end, I've never seen a wait even close to that unless it was completely down. Also the thing about "small prompts"...it sounds to me like you're overflowing context, they're returning an error, and somethings retrying.

Re: GPT-4 API General Availability

#106

Post by skilled »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

I can vouch on this. GPT4 API dies a lot if you use it for a big concurrent project. And of course it’s rate limited like crazy, with certain hours being so bad you can’t even run it for any business purpose.

Re: GPT-4 API General Availability

#107

Post by vbezhenar »

merpnderp wrote: post #64
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

It's the continued alignment with fine-tuning that's degrading its responses. You can apparently have it be nice or smart, but not both.

Why would someone care if its nice or not? It's an algorithm. You're using it to get output, not to get some psychology help.

Re: GPT-4 API General Availability

#108

Post by ericlewis »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

[flagged]

Re: GPT-4 API General Availability

#109

Post by anotherpaulg »

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I recently completed some benchmarks for code editing that compared the Feb (0301) and June (0613) versions of GPT-3.5 and GPT-4. I found indications that the June version of GPT-3.5 is worse than the Feb version.

https://aider.chat/docs/benchmarks.html

Re: GPT-4 API General Availability

#110

Post by jxy »

furyofantares wrote: post #78

Not a lot of talk of Whisper being available here. From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said. But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate. Given this is how OpenAI themselves use it (say your whole thing before getting…

You can run whisper.cpp locally in real time: https://github.com/ggerganov/whisper.cpp/tree/master/example...
Post reply on HN