Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

71–80 of 562 posts

Re: GPT-4 API General Availability

#71

Post by dr-detroit »

it_citizen wrote: post #59
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

[dead]

Re: GPT-4 API General Availability

#72

Post by merpnderp »

ClassicOrgin wrote: post #58
gamegoblin wrote: post #31

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…

I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.

You need to use the new OpenAI Functions API. It is absolutely bonkers at returning formatted results. I can get it to return a perfectly formatted query-graph a few levels deep.

Re: GPT-4 API General Availability

#73

Post by seizethecheese »

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

Re: GPT-4 API General Availability

#74

Post by PerryCox »

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

"Give me a daily schedule of a stoic hedonist" worked for me just now.

https://chat.openai.com/share/04c1dbc0-4890-447f-b5a5-7b1bc5...

Re: GPT-4 API General Availability

#75

Post by mike_hearn »

it_citizen wrote: post #59
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

ChatGPT is definitely more restricted than the API. Example:

https://news.ycombinator.com/item?id=36179783

Re: GPT-4 API General Availability

#76

Post by selalipop »

avindroth wrote: post #12
minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

I am building an extensive LLM-powered app, and had a chance to compare the two using the API. Empirically, I have found 3.5 to be fairly unusable for the app's use case. How are you evaluating the two models?

It depends on the domain, but chain of thought can get 3.5 to be extremely reliable, and especially with the new 16k variant

I built notionsmith.ai on 3.5: for some time I experimented with GPT 4 but the result was significantly worse to use because of how slow it became, going from ~15 seconds per generated output to a minute plus.

And you could work around that with things like streaming output for some use cases, but that doesn't work for chain of thought. GPT 4 can do some tasks without chain of thought that 3.5 required it for, but there are still many times where it improves the result from 4 dramatically.

For example, I leverage chain of thought in replies to the user when they're in a chat and that results in a much better user experience: It's very difficult to run into the default 'As a large language model' disclaimer regardless of how deeply you probe a generated experience when using it. GPT 4 requires the same chain of thought process to avoid that, but ends up needing several seconds per response, as opposed to 3.5 which is near-instant.

-

I suspect a lot of people are building things on 4 but would get better quality of output if they used more aspects of chain of thought and either settled for a slower output or moved to 3.5 (or a mix of 3.5 and 4)

Re: GPT-4 API General Availability

#77

Post by Terretta »

EnnioEvo wrote: post #10
minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

I have a startup of legal AI, the quality jump from GPT3.5 to GPT4 in this domain is straight mind-blowing, GPT3.5 in comparison is useless. But I see how in more conversational settings GPT3.5 can provide more appealing performance/price.

Same page.

So still waiting to be on the same 32 pages...

Re: GPT-4 API General Availability

#78

Post by furyofantares »

Not a lot of talk of Whisper being available here.

From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said.

But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate.

Given this is how OpenAI themselves use it (say your whole thing before getting feedback), I don't know that the API is set up to be able to mitigate that at all, but it would be really nice to have something closer to the responsiveness of on-device dictation with the quality of Whisper.

Re: GPT-4 API General Availability

#79

Post by londons_explore »

it_citizen wrote: post #59
hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process.

[2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s

[1]: https://arxiv.org/abs/2303.12712

Re: GPT-4 API General Availability

#80

Post by azemetre »

mike_hearn wrote: post #75
it_citizen wrote: post #59

Earlier quoted context omitted.

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

ChatGPT is definitely more restricted than the API. Example: https://news.ycombinator.com/item?id=36179783

That's disappointing, I thought ChatGPT WAS using the API. I mean what's the point of paying if you don't get similar levels of quality?
Post reply on HN