Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Audio Models

openai.fm

Post reply on HN
Print view View original

71–80 of 317 posts

Re: OpenAI Audio Models

#71

Post by minimaxir »

benjismith wrote: post #61

Is there way to get "speech marks" alongside the generated audio? FYI, Speech marks provide millisecond timestamp for each word in a generated audio file/stream (and a start/end index into your original source string), as a stream of JSONL objects, like this: {"time":6,"type":"word","start":0,"end":5,"value":"Hello"} {"time":732,"type":"word","start":7,"end":11,"value":"it's"} {"time":932,"type":"word","start":12,"en…

Passing the generated audio back to GPT-4o to ask for the structured annotations would be a fun test case.

Re: OpenAI Audio Models

#73

Post by forgotpasagain »

benjismith wrote: post #49

If I'm reading the pricing correctly, these models are SIGNIFICANTLY cheaper than ElevenLabs. https://platform.openai.com/docs/pricing If these are the "gpt-4o-mini-tts" models, and if the pricing estimate of "$0.015 per minute" of audio is correct, then these prices 85% cheaper than those of ElevenLabs. https://elevenlabs.io/pricing With ElevenLabs, if I choose their most cost-effectuve "Business" plan for $1100 per…

Almost everyone is cheaper than ElevenLabs though.

Re: OpenAI Audio Models

#74

Post by kuprel »

benjismith wrote: post #49

If I'm reading the pricing correctly, these models are SIGNIFICANTLY cheaper than ElevenLabs. https://platform.openai.com/docs/pricing If these are the "gpt-4o-mini-tts" models, and if the pricing estimate of "$0.015 per minute" of audio is correct, then these prices 85% cheaper than those of ElevenLabs. https://elevenlabs.io/pricing With ElevenLabs, if I choose their most cost-effectuve "Business" plan for $1100 per…

OpenAI doesn’t have voice cloning

Re: OpenAI Audio Models

#75

Post by crazygringo »

This is astonishing. I can type anything I want into the "vibe" box and it does it for the given text. Accents, attitudes, personality types... I'm amazed.

The level of intelligent "prosody" here -- the rhythm and intonation, the pauses and personality -- I wasn't expecting anything like this so soon. This is truly remarkable. It understands both the text and the prompt for how the speaker should sound.

Like, we're getting much closer to the point where nobody except celebrities are going to record audiobooks. Everyone's just going to pick whatever voice they're in the mood for.

Some fun ones I just came up with:

> Imposing villain with an upper class British accent, speaking threateningly and with menace.

> Helpful customer support assistant with a Southern drawl who's very enthusiastic.

> Woman with a Boston accent who talks incredibly slowly and sounds like she's about to fall asleep at any minute.

Re: OpenAI Audio Models

#79

Post by fixprix »

pzo wrote: post #63
fixprix wrote: post #50

Is this right? The current best TTS from OpenAI uses gpt-4o-audio-preview which is $2.50 input text, $80 output audio, the new gpt-4o-mini-tts is $0.60 input text, $12 output audio. An average 5x price reduction. Going the other way, transcribe with gpt-4o-audio-preview price was $40 input audio, $10 output text, the new gpt-4o-transcribe is $6 input audio and $10 output text. Like a 7x reduction on the input price.…

you can compare TTS pricing here: https://artificialanalysis.ai/text-to-speech Previous offering from OpenAI was $15 for TTS and $30 for TTS HD so not 5x reduction. This one is slighly cheaper but definitely more capable (if you need control vibe)

That's a really cool page thanks. Does it have stats for other languages?

In my experience the OpenAI TTS APIs were really bad, messing up all the time in foreign languages. Practically unusable for my use case. You'd have to use the gpt-4o-audio-preview to get anything close to passable, but it was expensive. Which is why I'm using Google TTS which is very fast, high quality, and provides first class support for almost every language.

I look forward to comparing it with this model, the price being the same is unfortunate as there's less incentive to switch. The transcribe price is cheaper than Google it looks like so that's worth considering.

Re: OpenAI Audio Models

#80

Post by dandiep »

jeffharris wrote: post #18

Hey, I'm Jeff and I was PM for these models at OpenAI. Today we launched three new state-of-the-art audio models. Two speech-to-text models—outperforming Whisper. A new TTS model—you can instruct it how to speak (try it on openai.fm!). And our Agents SDK now supports audio, making it easy to turn text agents into voice agents. We think you'll really like these models. Let me know if you have any questions here!

1) Previous TTS models had problems with major problems accents. E.g. a Spanish sentence could drift from a Spain accent to Mexican to American all within one sentence. Has this been improved and/or is it still a WIP?

2) What is the latency?

3) Your STT API/Whisper had MAJOR problems with hallucinating things the user didn't say. Is this fixed?

4) Whisper and your audio models often auto corrected speech, e.g. if someone made a grammatical error. Or if someone is speaking Spanish and inserted an English word, it would change the word to the Spanish equivalent. Does this still happen?

Post reply on HN