Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

371–380 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#371

Post by dharma1 »

vishal0123 wrote: post #82
minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…

Reckon they will (if not already) use 4bit or 8bit precision and may not need 175b params

Re: Introducing ChatGPT and Whisper APIs

#372

Post by userhacker »

BasilPH wrote: post #288
userhacker wrote: post #97

Earlier quoted context omitted.

I suggest you give revoldiv.com a try, We use whisper and other models together. You can upload very large files and get an hour long file transcription in less than 30 seconds. We use intelligent chunking so that the model doesn't lose context. We are looking to increase the limit even more in the coming weeks. It's also free to transcribe any video/audio with word level timestamps.

I just gave it a try, and the results are impressive! Do you also offer an API?

contact us at team@revoldiv.com and we are offering an API on a case by case basis

Re: Introducing ChatGPT and Whisper APIs

#373

Post by gerash »

soheil wrote: post #269

> Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December; we’re now passing through those savings to API users. Developers can now use our open-source Whisper large-v2 model in the API with much faster and cost-effective results. I'm really confused, I thought they were a non-profit. A non-profit to handle AI safety risks. Why does this read like a paragraph from a…

yeah I can't get over how easily they changed their branding from a non-profit AI safety to let's take over Google with the new Bing.

They come off as greedy to me and might very well try to get everyone locked in in order to milk them with Microsoft backing.

That said, they execute well, build good products and everyone loves more money so who am I to judge.

Re: Introducing ChatGPT and Whisper APIs

#374

Post by georgel »

osigurdson wrote: post #343

I'm not familiar with typical pricing but Whisper API at $0.006 / minute seems absurdly cheap!

I did a lot of research into this about 6 months ago, and the best price I could find/negotiate from the competition was 0.55/hr which included multi thousand dollar upfront commitments. This is 0.36/hr, and if you do a bit of setup work yourself you can bring it to about 0.09/hr. OpenAI offering hosted Whisper is a really good deal, and if you find it to be good for your application, then spending the time to host it yourself is perfect validation.

Re: Introducing ChatGPT and Whisper APIs

#375

Post by Auracle »

GaggiX wrote: post #182
ChildOfChaos wrote: post #117

Earlier quoted context omitted.

Stable diffusion might have a reasonable eco system around it, but automatic1111 was always around and 'completely crushes any competitors' is rather rich, Midjourney is still considered the standard as far as I was aware. I used both again recently and the difference was very clear, midjourney is leaps and bounds above anything else. Sure, stable diffusion has more control over the output, but the images are usually…

What models/LoRA you use with SD?

It doesn’t really matter. He’s right - Midjourney is leagues ahead as far as actually following your prompt and having it be aesthetically pleasing. I say this as someone who has made several Dreambooth and fine tuned models and has started to use Stable Diffusion in my work.

Now, if you happen to find or make a SD model that’s exactly what you’re looking for you’re in luck. I have no interest in it but it seems like all of the anime models work pretty well.

You obviously have a ton more control in SD, especially now with ControlNet. But if you want to see the Ninja Turtles surfing on Titan in the style of Rembrandt or something Midjourney will probably kick out something pretty good. Stable Diffusion won’t.

Re: Introducing ChatGPT and Whisper APIs

#376

Post by novaRom »

skybrian wrote: post #177
rvz wrote: post #70

Earlier quoted context omitted.

I have been saying this since the release of Stable Diffusion that OpenAI is going to struggle as soon as competitors release their models as open source especially when it surpasses GPT-3 and GPT-4. This is why OpenAI is rushing to bring their costs down and to make it close to free, However, Stable Diffusion is leading the race to the bottom and is already at the finish line, since no-one else would release their m…

Stable Diffusion isn’t free if you include the cost of the machine. Maybe you already have the hardware for some other reason, though? To compare total cost of ownership for a business, you need to compare using someone else’s service to running a similar service yourself. There’s no particular reason to assume OpenAI can’t do better at running a cloud service. Maybe someday you can assume end users have the hardware…

Ever heard about Federated Learning? This is the way it goes. Also, I do run training with no matrix multiplication, just 3-bit weights, addition in log space, slight accuracy degradation, but much faster CPU only training.

Re: Introducing ChatGPT and Whisper APIs

#377

Post by ALittleLight »

kkielhofner wrote: post #334
vishal0123 wrote: post #82

Earlier quoted context omitted.

> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…

But those A100s only come by eight and it’s speculated the model requires eight (VRAM). For a three year reservation that comes to over $96k/yr - to support one concurrent request.

What do you mean one concurrent request? Can't you have a huge batch size to basically support a huge number of concurrent requests?

e.g. Endpoint feeds a queue, queue fills a batch, batched results generate replies. You are simultaneously fulfilling many requests.

Re: Introducing ChatGPT and Whisper APIs

#378

Post by [deleted] »

LinuxBender wrote: post #99
warent wrote: post #57

Earlier quoted context omitted.

I'm pretty sure any system built via linear regression or similar is an opaque box even to the most experienced researchers. For example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron These are massive functions with billions of parameters that evolved over millions of computing years.

Adding to that, the human brain is incredibly complex and performs billions of functions. If a person says to me, "I love you" I should be able to ask them why they said that but it would probably be unfair to expect a detailed answer including all their environmental and genetic inputs, many of which they may not be aware of. If ChatGPT says it loves me, I not only expect the system to tell me why that was said but…

[deleted]

Re: Introducing ChatGPT and Whisper APIs

#379

Post by npunt »

whatshisface wrote: post #362
npunt wrote: post #338

Earlier quoted context omitted.

Yeah we're in an AI landgrab right now where at- or below-cost pricing is buying marketshare, lock-in, and underdevelopment of competitors. Smart move for them to pour money into it.

We have got to find a word for plans that are plainly harmful yet advantageous to their executors that's more descriptive than "smart..."

Agree. I didn't want to moralize, just wanted to point out it's a shrewd business move. It's rather anticompetitive, though that is hard to prove in such a dynamic market. Who knows, we may soon be calling it 'antitrust'.

Re: Introducing ChatGPT and Whisper APIs

#380

Post by arbuge »

So I had a question about how all this works under the hood. The GPT model is trained (on a massive dataset) and then deployed. How are they getting the additional product data from other sources like Instacart's retail partner locations and Shopify's store catalogs into it, so that it can output answers leveraging those? My understanding (perhaps incorrect) is that those weren't part of the dataset the model was initially trained on.

For example:

> Shop’s new AI-powered shopping assistant will streamline in-app shopping by scanning millions of products to quickly find what buyers are looking for

> This uses ChatGPT alongside Instacart’s own AI and product data from their 75,000+ retail partner store locations to help customers discover ideas for open-ended shopping goals

Post reply on HN