Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

531–540 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#531

Post by graderjs »

minimaxir wrote: post #87
noonething wrote: post #77

can't you do whisper stuff for free already?

Whisper large is a bit tricker to self-host, and the faster inference may be useful for certain applications.

Agree on that! Whisper large has big needs. But I didn't find the quality for English to be better than Medium. It just took longer. For most cases where audio is good quality, Small is all you need. Not much different to Medium. Only for really distorted (windy, loud background) audio were Medium and Large really good. But all models will fail beyond a certain point of extreme distortion.

If you don't believe me or want to know more check out my free app that uses Whisper Small, and (Whisper Tiny for Turbo mode): https://apps.apple.com/app/wisprnote/id1671480366

It uses VAD (voice activity detection) to reduce increased WER during silent or non-speech sections, and it's really fast! Runs locally on M1 just fine.

Re: Introducing ChatGPT and Whisper APIs

#532

Post by graderjs »

BasilPH wrote: post #273
rmorey wrote: post #88

Earlier quoted context omitted.

FWIW, AssemblyAI has great trasncript quality in my experience, and they support streaming: https://www.assemblyai.com/docs/walkthroughs#realtime-stream...

We're using AssemblyAI too, and I agree that their transcription quality is good. But as soon as Whisper supports world-level timestamps, I think we'll seriously consider switching as the price difference is large ($0.36 per hour vs $0.9 per hour).

Why do you need Word-level timestamps? I don't understand what that's for...

Re: Introducing ChatGPT and Whisper APIs

#533

Post by graderjs »

dgacmu wrote: post #501
sebzim4500 wrote: post #434

Earlier quoted context omitted.

Both of those prices strike me as quite high, given that Whisper can be run relatively quickly on commodity hardware. It's not like the bandwidth is significant either, it's just audio.

It's pretty great from my perspective. I've been creating little supplemental ~10 minute videos for my class (using descript; i should probably switch to OBS), and the built in transcription is both wonderful (that it has it at all and is easy to fix) and horrible (the number of errors is very high). I'd happily pay a dime to have a higher quality starting transcription that saves me 5 minutes of fixing...

Try my app: https://apps.apple.com/app/wisprnote/id1671480366

It has great quality transcription from video and audio (in English only sorry if that's not you!). Uses Whisper.cpp plus VAD to skip silent / non-speech sections which introduce errors normally. Give a try let me know what you think! :)

Re: Introducing ChatGPT and Whisper APIs

#534

Post by kierangill »

braingenious wrote: post #156

I hope this pricing impacts ChatGPT+ $20 is equivalent to what, 10,000,000 tokens? At ~750 words/1k tokens, that’s 7.5 million words per month, or roughly 250,000 words per day, 10,416 words per hour, 173 words per minute, every minute, 24/7. I uh, do not have that big of a utilization need. It’s kind of weird to vastly overpay

I use ChatGPT infrequently so the $20/month isn't worth it to me for pro.

I stood up a open source & login-free UI here: https://www.chatwithme.chat/

It accepts your API token and stores it in your browser. It does not have feature parity with ChatGPT but gives you the basics.

Re: Introducing ChatGPT and Whisper APIs

#535

Post by visarga »

dragonwriter wrote: post #451
CamperBob2 wrote: post #436

Earlier quoted context omitted.

No. The only way to a post-scarcity economy is by continuing to harness technology whenever and wherever possible. There has been, and will be, no point in time at which the optimal strategy is "Stop" or "Wait" or "What about my job?"

> The only way to a post-scarcity economy There is no way to a post-scarcity economy, the phrase is a contradiction, and scarcity is an inescapable consequence of human desire.

There's no scarcity of information and media.

Re: Introducing ChatGPT and Whisper APIs

#536

Post by Too »

somenameforme wrote: post #526
freeqaz wrote: post #236

Earlier quoted context omitted.

It's also worth mentioning that, because Microsoft is an investor, they're likely getting these at cost or subsidized. OpenAI doesn't have to make money right away. They can lose a small bit of money per API request in exchange for market share (preventing others from disrupting them). As the cost of GPUs goes down, or they develop at ASIC or more efficient model, they can keep their pricing the same and then make mo…

Who will they be making money from? OpenAI is looking for companies willing to: - tolerate the current state of the chatbots - tolerate the high per-query latency - tolerate having all queries sent to OpenAI - tolerate OpenAI [presumably] having 0 liability for ChatGPT just randomly hallucinating inappropriate nonsense - be willing to pay a lot of money for the above I'm kind of making an assumption on that last poin…

Sadly, it will likely mostly be used to generate endless streams of SEO spam. Not for interactive use.

Re: Introducing ChatGPT and Whisper APIs

#537

Post by visarga »

thedangler wrote: post #195
bicx wrote: post #183

Earlier quoted context omitted.

You can do this to an extent via fine-tuning, but you will need to do so via one of the other GPT-3 models rather than the ChatGPT API model (`GPT-3.5-turbo`). The latter is not available for fine-tuning.

Does openAI have models for data to train it with.. data like pricing, locations, products .... Or would I have to use something else? The reason I ask because companies are using it with their own data, like Shopify... sooo it has to be trained somehow.

Yes, you bring your own data, or you generate data with the model, that works too. I have generated training data with GPT3 and trained smaller HuggingFace models on it.

Re: Introducing ChatGPT and Whisper APIs

#538

Post by visarga »

habitue wrote: post #166

Speculation: GPT-turbo is a new chinchilla optimal model with the equivalent capabilities as GPT-3.5. So it's literally just smaller, faster and cheaper to run. The reason I don't think it's just loss-leading is that they made it faster too. That heavily implies a smaller model.

Not Chinchilla-optimal but Inference-optimal. Chinchilla optimality was related to the training budget and is of interest to researchers who produce mainly demos. Inference optimality includes the inference costs and is of interest in real deployments to millions of users. It is worth to pay more for training to reduce inference costs, so they probably went even further than Chinchilla.

Re: Introducing ChatGPT and Whisper APIs

#539

Post by ar9av »

minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

Pricing of this model seems less per token level but you have to send the entire conversation each time, and the tokens you will be billed for include both those you send and the API's response (which you are likely to append to the conversation and send back to them, getting billed again and again as the conversation progresses). By the time you've hit the 4K token limit of this API, there will have been a bunch of back and forth - you'll have paid a lot more than 4K * 0.002/1K for the conversation.

Re: Introducing ChatGPT and Whisper APIs

#540

Post by TechBro8615 »

rwill128 wrote: post #502
TechBro8615 wrote: post #428

Earlier quoted context omitted.

Thank you for posting a link to an image instead of polluting the future training data of GPT-4 with the output of GPT-3 :) I wish more people would do this. I'm getting pretty sick of the walls of text.

That pollution is inevitable, why delay it? It's a technical problem they should be able to solve, and if they can't, then they're revealing the weakness of their methods and the shortcomings of their so-called AI. It's absolutely ridiculous to expect the entire internet to adopt some kind of hygiene practices when it comes to text from GPT tools simply for the sake of making the training process slightly easier for…

No, that's just a bonus. I just personally find the walls of texts in HN comments to be necessary.

I saw someone on here refer to it as "listening to someone describe their dreams." I pretty much agree with that.

Post reply on HN