Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

401–410 of 562 posts

Re: GPT-4 API General Availability

#401

Post by selcuka »

ec109685 wrote: post #339
selcuka wrote: post #330

Earlier quoted context omitted.

> Microsoft is giving that for free but I assume they're paying OpenAI for it. Yeah, but Microsoft already gets 75% of the profits OpenAI makes, it's not the same price for them as the rest of us.

It’s the exactly the same. If they could make 75 cents selling the compute to someone else for $1 versus not making it providing the Bing chat service, that is 75 cents they lose.

Why do you assume that the same amount of computing power would be used by someone else? There are only so many customers. You can't magically start selling more compute if you stop using it yourself.

Re: GPT-4 API General Availability

#402

Post by RossBencina »

PostOnce wrote: post #372
andai wrote: post #273

Earlier quoted context omitted.

Can you elaborate on scary smart and fast? It's been a month or two since I've tried but the results were depressingly slow and useless for more or less every task I tried. Every time a model is claimed to be "90% of GPT-3" I get excited and every time it's very disappointing. (On that note, after using GPT-4, GPT-3 now seems disappointing almost every time I interact with it.)

Different quantizations can give you a big speedup if you've had "depressingly slow" issues. Even the slowest ones (that fit in RAM) will run at basically interactive speed, not instant, but also not "email speed". I have a laptop with a 2018 CPU and I'm working with them just fine. Text generation style instead of chat style is another avenue that makes the feedback time not so annoying for a developer. at 100ms/tok…

How are you running inference? GPU or CPU? I'm trying to use GPT4All (ggml-based) on 32 cores of E5-v3 hardware and even the 4GB models are depressingly slow as far as I'm concerned (i.e. slower than the GPT4 API, which is barely usable for interactive work). I'd be much obliged if you could point me at a specific quantized model on HF that you think is "fast" and I'll download it and try it out.

Re: GPT-4 API General Availability

#403

Post by raincole »

system2 wrote: post #234
superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

I built my entire app on text-davinci-003. It is the best writer so far. Do you think gpt3.5 turbo instruct won't be the same?

> In the coming weeks, we will reach out to developers who have recently used these older models, and will provide more information once the new completion models are ready for early testing.

I guess they'll give you early access to it.

Re: GPT-4 API General Availability

#404

Post by raincole »

jiggawatts wrote: post #283

The drip-feeding seems crazy to me. Open AI is undermining their reputation by forcing almost everybody to use the older, lower-quality models. Even if customers are willing to pay for GPT 4, they're being told to wait at the back of the line. Wait for what!? Christmas? When we can open our presents and have a GPT 4 inside? It's like they took a leaf from Google's "how to guarantee the failure of a new product" marke…

> Wait for what!? Christmas?

Infrastructure.

> It's like they took a leaf from Google's "how to guarantee the failure of a new product" marketing.

Yeah, an infamous guaranteed failure: GPT-4. (canned laughter)

Re: GPT-4 API General Availability

#405

Post by zirgs »

claytonjy wrote: post #190
ronsor wrote: post #186

Earlier quoted context omitted.

If you're finetuning your own model, the closed models being "incredibly higher quality" is probably less relevant.

That's how we all want it to work, but the reality today is that GPT-4 is better at almost anything than a fine-tuned version of any other model. It's somewhat rare to have a task and good enough dataset that you can finetune something else to be close enough in quality to GPT-4 for your task.

GPT-4 is still heavily censored and will simply refuse to talk about many "problematic" things. How is that better than a completely uncensored model?

Re: GPT-4 API General Availability

#406

Post by LeoPanthera »

superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

Everyone who complains about being "censored" never gives examples.

Re: GPT-4 API General Availability

#407

Post by alvah »

minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

I think this is very very use-case dependent, and your use case != everyone's use case. In my experience, GPT-4 is night and day better than 3.5 turbo for almost everything I use OpenAI for.

Re: GPT-4 API General Availability

#408

Post by system2 »

raincole wrote: post #403
system2 wrote: post #234

Earlier quoted context omitted.

I built my entire app on text-davinci-003. It is the best writer so far. Do you think gpt3.5 turbo instruct won't be the same?

> In the coming weeks, we will reach out to developers who have recently used these older models, and will provide more information once the new completion models are ready for early testing. I guess they'll give you early access to it.

Thanks!

Re: GPT-4 API General Availability

#409

Post by ignoramous »

bredren wrote: post #385
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Running an LLM locally and paying for access to OpenAI are two separate concerns. But to address both: is it very relevant what LLM you use right now? Local or hosted, openAI or other? It seems like the interface has converged around chat-based prompts. New ideas for tuning or improving the efficiency of foundational models are published almost every week. If one wants to build a product on top of of generative AI, w…

> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?

This, infact, might be a better way to do inference anyway: https://twitter.com/Francis_YAO_/status/1675967988925710338

> So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances?

Depends. There are signs that folks are buying into GPT-specific APIs (like function calls) which may not be as easy to migrate away from.

Re: GPT-4 API General Availability

#410

Post by ignoramous »

kordlessagain wrote: post #395
PostOnce wrote: post #253

Earlier quoted context omitted.

OpenLLaMA is though. https://github.com/openlm-research/open_llama All of these are surmountable problems. We can beat OpenAI. We can drain their moat.

> We can drain their moat. I've got an AI powered sump pump if you need it.

They most certainly don't need / deserve the snark, to be sure, on hacker news of all places.
Post reply on HN