Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

201–210 of 562 posts

Re: GPT-4 API General Availability

#201

Post by john2x »

PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options.

I have an old laptop with 16GB RAM and no GPU. Can I run these models?

Re: GPT-4 API General Availability

#203

Post by stavros »

superkuh wrote: post #169
H8crilA wrote: post #162

Earlier quoted context omitted.

The $5-$10 is probably the reason why they're killing those endpoints.

I don't get it? text-davinci-003 is the most expensive model per token. It's just that running IRC bots isn't exactly high volume.

"Most expensive" doesn't mean "highest margin", though.

Re: GPT-4 API General Availability

#204

Post by popinman322 »

ravenstine wrote: post #30
minimaxir wrote: post #22

Earlier quoted context omitted.

The initial gpt-3.5-turbo was flakey and required significant prompt engineering. The updated gpt-3.5-turbo-0613 fixed all the issues I had even after stripping out the prompt engineering.

It's definitely gotten better, but yeah, it really doesn't reliably support what I'm currently working on. My project takes transcripts from YouTube, which don't have punctuation, splits them up into chunks, and passes each chunk to GPT-4 telling it to add punctuation with paragraphs. Part of the instructions includes telling the model that, if the final sentence of the chunk appears incomplete, to just try to comple…

I've done exactly this for another project. I'd recommend grabbing an open source model and fine-tuning on some augmented data in your domain. For example: I grabbed tech blog posts, turned each post into a collection of phonemes, reconstructed the phonemes into words, added filler words, and removed punctuation+capitalization.

Re: GPT-4 API General Availability

#205

Post by purplecats »

renewiltord wrote: post #112
crancher wrote: post #90

Earlier quoted context omitted.

API call responsiveness to the GPT-4 model varies hugely throughout the day. The #1 datapoint in measured responsiveness is slowdown associated with lunch-time use as noon sweeps around the globe.

Thank you for the response, I should have been clearer. I meant performance as an LLM. Essentially, I am concerned that they are quietly nerfing the tool. The Chat interface is now very verbose and constantly warning me about "we should always do this and that" which is bloody exasperating when I'm just trying to get things done. I made up an example here to illustrate, but it's just very annoying because sometimes i…

yeah, its annoying and you have to foot the bill for it.

looking at your sample and using character count as a rough proxy for tokens, (465/(1581-465))*100 means they added ~42% token count cost to your response explicitly adding caveats which you dont want. fun!

Re: GPT-4 API General Availability

#206

Post by simonw »

flangola7 wrote: post #184
alpark3 wrote: post #174

>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge…

They address that, OpenAI will cover the cost of re-training on the new models, and the old models don't discontinue until next year.

Did they say they would cover the cost of fine-tuning again? I saw them say they would cover the cost of recalculating embeddings, but I didn't see the bit about fine-tuning costs.

On fine-tuning:

> We will be providing support to users who previously fine-tuned models to make this transition as smooth as possible.

On embeddings:

> We will cover the financial cost of users re-embedding content with these new models.

Re: GPT-4 API General Availability

#207

Post by stavros »

codazoda wrote: post #178
ftxbro wrote: post #176

> "Starting today, all paying API customers have access to GPT-4." OK maybe I'm stupid but I am a paying OpenAI API customer and I don't have it yet. I see: gpt-3.5-turbo-16k gpt-3.5-turbo gpt-3.5-turbo-16k-0613 gpt-3.5-turbo-0613 gpt-3.5-turbo-0301 I don't see any gpt-4 Edit: Probably my problem is that I upgraded to paid API account within the last month, so I'm not technically a "paying API customer" yet according…

> Today all existing API developers with a history of successful payments can access the GPT-4 API with 8K context. We plan to open up access to new developers by the end of this month, and then start raising rate-limits after that depending on compute availability. Same for me. I signed up only a few days ago and was excited to switch to "gpt-4" but I haven't paid the first bill (save the $5 capture) so I probably h…

Interesting, I did exactly the same (with the same name), but with GPT-4 support as well:

https://www.pastery.net/ccvjrh/

It also does streaming, so it live-prints the response as it comes.

Re: GPT-4 API General Availability

#209

Post by mrinterweb »

gamegoblin wrote: post #69
ClassicOrgin wrote: post #58

Earlier quoted context omitted.

I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.

I'm assuming they will price it the same as normal gpt-3.5-turbo. I won't use it if it's more than 2x the price of turbo, because I can usually get turbo to do what I want, it just takes more tokens sometimes. Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.

From what I can find, pricing of GPT-4 is roughly 25x that of 3.5 turbo.

https://openai.com/pricing

https://platform.openai.com/docs/deprecations/

Re: GPT-4 API General Availability

#210

Post by browningstreet »

whalesalad wrote: post #166
seizethecheese wrote: post #73

Earlier quoted context omitted.

Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.

It definitely got nerfed.

I've never seen "nerf" used colloquially and today i've seen it at least a half-dozen times across various sites. Y'all APIs?
Post reply on HN