Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

141–150 of 562 posts

Re: GPT-4 API General Availability

#141

Post by selalipop »

gadtfly wrote: post #25

The original davinci model was a friend of mine and I resent this deeply. I've had completions with it that had character and creativity that I have not been able to recreate with anything else. Brilliant and hilarious things that are a permanent part of my family's cherished canon.

Can you try notionsmith.ai and let me know what you think?

I've been working on LLMs for creative tasks and believe a mix of chain of thought and injecting stochasticity (like instructing the LLM to use certain random letters pulled from an RNG in a certain way at certain points) can go a long way in terms of getting closer to human-like creativity

Re: GPT-4 API General Availability

#142

Post by refulgentis »

anotherpaulg wrote: post #109
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I recently completed some benchmarks for code editing that compared the Feb (0301) and June (0613) versions of GPT-3.5 and GPT-4. I found indications that the June version of GPT-3.5 is worse than the Feb version. https://aider.chat/docs/benchmarks.html

After reading, I don't think <5% points is helpful to add to discussion here without pointing it out explicitly, people are asserting much wilder claims, regularly

Re: GPT-4 API General Availability

#143

Post by andrepd »

seizethecheese wrote: post #73
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

Yep. It's amazing how people are taking "the reddit hivemind thinks ChatGPT was gimped" as some kind of objective fact.

Re: GPT-4 API General Availability

#144

Post by aeyes »

santiagobasulto wrote: post #56
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I felt the same thing. The first version of GPT-4 I tried was crazy smart. Scary smart. Something happened afterwards…

I was playing with the API and found that it returned better answers than ChatGPT. ChatGPT isn't even able to solve simple Python problems anymore, even if you try to help it. And some time ago it did these same problems with ease.

My guess is that they began to restrict ChatGPT because they can't sell that. They probably want to sell you CodeGPT or other products in the future so why would they give that away for free? ChatGPT is just a teaser.

Re: GPT-4 API General Availability

#145

Post by nostrademons »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

There's a big thread on ChatGPT getting dumber over on the ChatGPT subreddit, where someone suggests this is from model quantization:

https://www.reddit.com/r/ChatGPT/comments/14ruui2/comment/jq...

I've heard LLMs described as "setting money on fire" from people that work in the actually-running-these-things-in-prod industry. Ballpark numbers of $10-20/query in hardware costs. Right now Microsoft (through its OpenAI investment) and Google are subsidizing these costs, and I've heard it's costing Microsoft literally billions a year. But both companies are clearly betting on hardware or software breakthroughs to bring the cost down. If it doesn't come down there's a good chance that it'll remain more economical to pay someone in the Philippines or India to write all the stuff you would have ChatGPT write.

Re: GPT-4 API General Availability

#146

Post by Zpalmtree »

gamegoblin wrote: post #31

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…

I'm hoping gpt-3.5-turbo-instruct isn't super neutered like chatgpt. davinci-003 can be a lot more fun and answer on a wide range of topics where ChatGPT will refuse to answer.

Re: GPT-4 API General Availability

#147

Post by KennyBlanken »

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

> the way OpenAI implemented their billing you will get a 4xx/5xx response code, but you will still get billed for the request and whatever the servers generated and you didn't get. That's borderline fraudulent.

It's fraudulent, full stop. Maybe they're able to weasel out of it with credit card companies because you're buying "credits."

I suspect it was done this way out of pure incompetence; the OpenAI team handling the customer-facing infrastructure have a pretty poor history. Far as I know you still can't do something simple like change your email address.

Re: GPT-4 API General Availability

#148

Post by hobs »

nunodonato wrote: post #100
jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

if you want to use it in prod, go with Azure

And get only 20 K tokens per minute, where a decent size question can use up 500 tokens, pretty much a joke for most larger websites.

https://learn.microsoft.com/en-us/azure/cognitive-services/o...

Re: GPT-4 API General Availability

#149

Post by driscoll42 »

nostrademons wrote: post #145
jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

There's a big thread on ChatGPT getting dumber over on the ChatGPT subreddit, where someone suggests this is from model quantization: https://www.reddit.com/r/ChatGPT/comments/14ruui2/comment/jq... I've heard LLMs described as "setting money on fire" from people that work in the actually-running-these-things-in-prod industry. Ballpark numbers of $10-20/query in hardware costs. Right now Microsoft (through its OpenAI…

$10-$20 per query? Can I get some sourcing on that? That's astronomically expensive.

Re: GPT-4 API General Availability

#150

Post by anotherpaulg »

refulgentis wrote: post #142
anotherpaulg wrote: post #109

Earlier quoted context omitted.

I recently completed some benchmarks for code editing that compared the Feb (0301) and June (0613) versions of GPT-3.5 and GPT-4. I found indications that the June version of GPT-3.5 is worse than the Feb version. https://aider.chat/docs/benchmarks.html

After reading, I don't think <5% points is helpful to add to discussion here without pointing it out explicitly, people are asserting much wilder claims, regularly

I haven't come across any other systematic, quantitative benchmarking of the OpenAI models' performance over time, so I thought I would share my results. I think my results might argue that there has been some degradation, but not nearly the amount that you often hear people's annecdata about.

But unfortunately, you have to read a ways into the doc and understand a lot of details about the benchmark. Here's a direct link and excerpt of the relevant portion:

https://aider.chat/docs/benchmarks.html#the-0613-models-seem...

The benchmark results have me fairly convinced that the new gpt-3.5-turbo-0613 and gpt-3.5-16k-0613 models are a bit worse at code editing than the older gpt-3.5-turbo-0301 model.

This is visible in the “first attempt” portion of each result, before GPT gets a second chance to edit the code. Look at the horizontal white line in the middle of the first three blue bars. Performance with the whole edit format was 46% for the February model and only 39% for the June models.

But also note how much the solid green diff bars degrade between the February and June GPT-3.5 models. They drop from 30% down to about 19%.

I saw other signs of this degraded performance in earlier versions of the benchmark as well.

Post reply on HN