Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

81–90 of 562 posts

Re: GPT-4 API General Availability

#81

Post by stuckkeys »

sashank_1509 wrote: post #52
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I feel like it's code generation abilities have also been nerfed. In the past I got almost excellent code from GPT-4, somehow these days I need multiple prompts to get the code I want from GPT-4.

Not nerfed. They will sell a different tier service to assist with coding. Coming soon. Speculating ofc.

Re: GPT-4 API General Availability

#82

Post by colordrops »

seizethecheese wrote: post #73
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

It's both. OpenAI is obviously tuning the model for both computational resource constraints as well as "alignment". It's not an either-or.

Re: GPT-4 API General Availability

#83

Post by we_never_see_it »

It's funny how OpenAI just shattered Google's PR stunts. Google wanted everyone to believe they are leading in AI by winning some children's games. Everyone thought that was the peak of AI. Enter OpenAI and Micorsoft. Microsoft and OpenAI have showed the humanity what true AI looks like. Like most people on HN I cannot wait to see the end of Google, the end of evil.

Re: GPT-4 API General Availability

#84

Post by superkuh »

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison.

This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire under me to get the 65B llama models working locally.

Re: GPT-4 API General Availability

#85

Post by londons_explore »

sashank_1509 wrote: post #52
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I feel like it's code generation abilities have also been nerfed. In the past I got almost excellent code from GPT-4, somehow these days I need multiple prompts to get the code I want from GPT-4.

In the API, you can select to use the 14th March 2023 version of GPT-4, and then compare them side by side.

Re: GPT-4 API General Availability

#86

Post by senko »

ClassicOrgin wrote: post #58
gamegoblin wrote: post #31

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…

I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.

With the latest 3.5-turbo, you can try forcing it to call your function with a well-defined schema for arguments. If the structure is not overly complex, this should work.

Re: GPT-4 API General Availability

#87

Post by it_citizen »

londons_explore wrote: post #79
it_citizen wrote: post #59

Earlier quoted context omitted.

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s [1]: https://arxiv.org/abs/2303.12712

Thanks, that’s interesting.

Noobie follow up question: Should we put any trust into “Sparks of intelligence” I thought it was regarded as a Microsoft marketing piece, not a serious paper.

Re: GPT-4 API General Availability

#88

Post by londons_explore »

it_citizen wrote: post #87
londons_explore wrote: post #79

Earlier quoted context omitted.

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s [1]: https://arxiv.org/abs/2303.12712

Thanks, that’s interesting. Noobie follow up question: Should we put any trust into “Sparks of intelligence” I thought it was regarded as a Microsoft marketing piece, not a serious paper.

The data presented is true... The text might be rather exaggerated/unscientific/marketing...

Also notable that the team behind that paper wasn't involved in designing/building the model, but they did get access to prerelease versions.

Re: GPT-4 API General Availability

#89

Post by jwr »

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4.

The API will make you wait up to 10 minutes, and then time out. What's worse, it will time out between their edge servers (cloudflare) and their internal servers, and the way OpenAI implemented their billing you will get a 4xx/5xx response code, but you will still get billed for the request and whatever the servers generated and you didn't get. That's borderline fraudulent.

Meanwhile, their status page will happily show all green, so don't believe that. It seems to be manually updated and does not reflect the truth.

Could it be that it works better in another region? Could it be just my region that is affected? Perhaps — but I won't know, because support is non-existent and hidden behind a moat. You need to jump through hoops and talk to bots, and then you eventually get a bot reply. That you can't respond to.

My support requests about being charged for data I didn't have a chance to get have been unanswered for more than 5 weeks now.

There is no way to contact OpenAI, no way to report problems, the API sometimes kind-of works, but mostly doesn't, and if you comment in the developer forums, you'll mostly get replies from apologists that explain that OpenAI is "growing quickly". I'd say you either provide a production paid API or you don't. At the moment, this looks very much like amateur hour, and charging for requests that were never fulfilled seems like a fraud to me.

So, consider carefully whether you want to build against all that.

Re: GPT-4 API General Availability

#90

Post by crancher »

renewiltord wrote: post #14

Has anyone been able to come up with a way to keep track of GPT-4 performance over time? I'm told that the API is explicit about changes to models and that the Chat interface is not.

API call responsiveness to the GPT-4 model varies hugely throughout the day. The #1 datapoint in measured responsiveness is slowdown associated with lunch-time use as noon sweeps around the globe.
Post reply on HN