Live data from Hacker News

OpenAI Audio Models

openai.fm

101–110 of 317 posts

Re: OpenAI Audio Models

#101
post #46
post #7

Recommended input for anyone trying this out: Voice: Onyx Vibe: Heavy german accent, doing an Arnold Schwarzenegger impression, way over the top for comedic effect. Deep booming voice, uses pauses for dramatic effect.

Weird; trying exactly this, and every time I stop and play again, I get a totally different voice. One of them (if I'm not mistaken) was cod-Russian.

I also find this strange, and I wonder if I can get a consistent voice out of this. If using the api with a vibe/instructions for a back and forth, will it be consistent? This example app they provide implies no?

Re: OpenAI Audio Models

#102

Earlier quoted context omitted.

Hi Jeff, Thanks for updating the TTS endpoint! I was literally about to have to make a workaround with the chat completions endpoint with a hit and hope the transcription matches strategy... as it was the only way to get the updated voice models. Curious.. is gpt-4o-mini-tts the equivilant of what is/was gpt-4o-mini-audio-preview for chat completions? Because in timing tests it takes around 2 seconds to return a shor…

It's a slightly better model for TTS. With extra training focusing on reading the script exactly as written. e.g. the audio-preview model when given instruction to speak "What is the capital of Italy" would often speak "Rome". This model should be much better in that regard = No plans to have localized voice models, but we do want to bring expand the menu of voices with voices that are best at different accents

Great to hear thanks. My favorite was "I would like you to repeat the following in an Australian accent: Hi there, welcome to Sydney." which was more often than not swapping "Hi there" for "G'day"!

Re: OpenAI Audio Models

#103
post #99

Hey, I'm Jeff and I was PM for these models at OpenAI. Today we launched three new state-of-the-art audio models. Two speech-to-text models—outperforming Whisper. A new TTS model—you can instruct it how to speak (try it on openai.fm!). And our Agents SDK now supports audio, making it easy to turn text agents into voice agents. We think you'll really like these models. Let me know if you have any questions here!

Hey Jeff, maybe you could improve the TTS that is currently in the OpenAI web and phone apps. When I set it to read numbers in Romanian it slurs digits. This also happens sometimes with regular words as well. I hope you find resources for other languages than English.

thanks for flagging ... number fidelity (especially on languages that are unfortunately less represented in training data) is still something we're working to improve

Re: OpenAI Audio Models

#105
I'm surprised at how poor this is at following a detailed prompt.

It seems capable of generating a consistent style, and so in that sense quite useful. But if you want (say) a regional UK accent it's not even close.

I also find it confusing you have to choose a voice. Surely that's what the prompt should be for, especially when the voices have such abstract names.

I mean, it's still very impressive when you stand back a bit, but feels a bit half baked

Example: Voice: Thick and hearty, with a slow, rolling cadence—like a lifelong Somerset farmer leaning over a gate, chatting about the land with a mug of cider in hand. It’s warm, weathered, and rich, carrying the easy confidence of someone who’s seen a thousand harvests and knows every hedgerow and rolling hill in the county.

Tone: Friendly, laid-back, and full of rustic charm. It’s got that unhurried quality of a man who’s got time for a proper chinwag, with a twinkle in his eye and a belly laugh never far away. Every sentence should feel like it’s been seasoned with fresh air, long days in the fields, and a lifetime of countryside wisdom.

Dialect: Classic West Country, with broad vowels, softened consonants, and that unmistakable rural lilt. Words flow together in an easy drawl, with plenty of dropped "h"s and "g"s. "I be" replaces "I am," and "us" gets used instead of "we" or "me." Expect plenty of "ooh-arrs," "proper job," and "gurt big" sprinkled in naturally.

Re: OpenAI Audio Models

#106

Hey, I'm Jeff and I was PM for these models at OpenAI. Today we launched three new state-of-the-art audio models. Two speech-to-text models—outperforming Whisper. A new TTS model—you can instruct it how to speak (try it on openai.fm!). And our Agents SDK now supports audio, making it easy to turn text agents into voice agents. We think you'll really like these models. Let me know if you have any questions here!

Does whispering work? I could not get it to work when I tried it

Should do! here's an example https://www.openai.fm/#4a5a82db-faea-4f80-813c-3131902c2458

Re: OpenAI Audio Models

#107

This is astonishing. I can type anything I want into the "vibe" box and it does it for the given text. Accents, attitudes, personality types... I'm amazed. The level of intelligent "prosody" here -- the rhythm and intonation, the pauses and personality -- I wasn't expecting anything like this so soon. This is truly remarkable. It understands both the text and the prompt for how the speaker should sound. Like, we're g…

> Everyone's just going to pick whatever voice they're in the mood for.

I can't say I've ever had this impulse. Also, to point out the obvious, there's little reason to pay for an audiobook if there's no human reading it. Especially if you already bought the physical text.

Re: OpenAI Audio Models

#108
post #37

Hey, I'm Jeff and I was PM for these models at OpenAI. Today we launched three new state-of-the-art audio models. Two speech-to-text models—outperforming Whisper. A new TTS model—you can instruct it how to speak (try it on openai.fm!). And our Agents SDK now supports audio, making it easy to turn text agents into voice agents. We think you'll really like these models. Let me know if you have any questions here!

Are these models downloadable, like whisper? What’s the minimum hardware for running them? Would they run on a raspberry pi? Or a smartphone?

not open source at this time. unfortunately they're much to large to run on normal consumer hardware

Re: OpenAI Audio Models

#110
post #99

Earlier quoted context omitted.

Hey Jeff, maybe you could improve the TTS that is currently in the OpenAI web and phone apps. When I set it to read numbers in Romanian it slurs digits. This also happens sometimes with regular words as well. I hope you find resources for other languages than English.

thanks for flagging ... number fidelity (especially on languages that are unfortunately less represented in training data) is still something we're working to improve

Actually even the new model does it. I put it read "12345 54321" and it read "2346 5321". So it both skips and hallucinates digits. This could be dangerous if it is used to read some news article or important text with numbers.
Post reply on HN