Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Audio Models

openai.fm

Post reply on HN
Print view View original

251–260 of 317 posts

Re: OpenAI Audio Models

#251

Post by tough »

wholinator2 wrote: post #199
lukebuehler wrote: post #194

Earlier quoted context omitted.

I din't think of that, interesting idea. What I'm focusing right now is long-form content for more offline-ish listening, but maybe a plugin could work to load longer texts, but I'm not working on a screen reader atm.

Do you know if there's any offerings today that can read math? Like speak an equation the way a human would? It's something I've been thinking about a long time and would be an essential feature for me (the only things i read are physics)

I saw a small model trained on outputting currency aware text from decimals/integers

i wonder if you could make a similar -narrow- lora finetune to train a model to output human readable text from say latext formulas with a good data set to train on

Re: OpenAI Audio Models

#252

Post by oidar »

jeffharris wrote: post #208
oidar wrote: post #204

Earlier quoted context omitted.

Any plans to offer speech to speech models which keep prosody, intonation, and timing intact? ElevenLabs is getting expensive for this.

we'll keep expanding these GPT-4o based models with more controls. Is the main feature missing we're missing custom voices?

No, not custom voices - but voices that can be influenced by a recording. As in, a male voice actor records a part, and the model transforms it to a female part - keeping all the prosody, intonation and timing in the original recording. This would allow one voice actor to do many roles.

Re: OpenAI Audio Models

#253

Post by throwup238 »

borgdefenser wrote: post #172
crazygringo wrote: post #75

This is astonishing. I can type anything I want into the "vibe" box and it does it for the given text. Accents, attitudes, personality types... I'm amazed. The level of intelligent "prosody" here -- the rhythm and intonation, the pauses and personality -- I wasn't expecting anything like this so soon. This is truly remarkable. It understands both the text and the prompt for how the speaker should sound. Like, we're g…

I am always listening to audio books but they are no good anymore after playing with this for 2 minutes. I am never really in the mood for a different voice. I am going to dial in the voice I want and only going to want to listen with that voice. This is so awesome. So many audio books have been ruined by the voice actor for me. What sticks out in my head is The Book of Why by Judea Pearl read by Mel Foster. Brutal.…

The ElevenReader app from ElevenLabs has been able to do that for a while now and they’ve licensed some celebrity voices like Burt Reynolds. You can use the browser share function to send it a webpage to read or upload a PDF or epub of a book.

It’s far from perfect though. I’m listening to Shattered Sword (about the battle of midway) which has lots of academic style citations so every other sentence or paragraph ends with it spelling out the citation number like “end of sentence dot one zero”, it’ll often mangle numbers like “1,000 pound bomb” becomes “one zero zero zero pound bomb”, and it tries way too hard to expand abbreviations so “Operation AL” becomes “Operation Alabama” when it’s really short for Aleutian Islands.

Re: OpenAI Audio Models

#254

Post by noahlt »

goshx wrote: post #216
oidar wrote: post #202

Earlier quoted context omitted.

ElevenLabs is the only one offering speech to speech generation where the intonation, prosody, and timing is kept intact. This allows for one expressive voice actor to slip into many other voices.

OpenAI’s Realtime speech to speech is far superior than ElevenLabs.

What ElevenLabs and OpenAI call “speech to speech” are completely different.

ElevenLabs’ takes as input audio of speech and maps it to a new speech audio that sounds like a different speaker said it, but with the exact same intonation.

OpenAI’s is an end-to-end multimodal conversational model that listens to a user speaking and responds in audio.

Re: OpenAI Audio Models

#255

Post by tkgally »

tkgally wrote: post #241

I just tested the "gpt-4o-mini-tts" model on several texts in Japanese, a particularly challenging language for TTS because many character combinations are read differently depending on the context. The produced speech was quite good, with natural intonation and pronunciation. There were, however, occasional glitches, such the word 現在 genzai “now, present” read with a pause between the syllables ( gen ... zai ) and t…

Self-correction: "... good enough for language users" --> "good enough for language learners."

Re: OpenAI Audio Models

#256

Post by SXX »

com2kid wrote: post #185
benjismith wrote: post #49

If I'm reading the pricing correctly, these models are SIGNIFICANTLY cheaper than ElevenLabs. https://platform.openai.com/docs/pricing If these are the "gpt-4o-mini-tts" models, and if the pricing estimate of "$0.015 per minute" of audio is correct, then these prices 85% cheaper than those of ElevenLabs. https://elevenlabs.io/pricing With ElevenLabs, if I choose their most cost-effectuve "Business" plan for $1100 per…

Elevenlabs is an ecosystem play. They have hundreds of different voices, legally licensed from real people who chose to upload their voice. It is a marketplace of voices. None of the other major players is trying to do that, not sure why.

Going with this would mean AI companies suppose to pay for something like voices or other training data.

It's far better to just steal it all and ask government for exception.

Re: OpenAI Audio Models

#257

Post by pests »

swyx wrote: post #200
ComputerGuru wrote: post #11

It would be much more convenient to use if changing the voice model worked on the fly, without having to stop and start the audio.

convenient why?

It breaks the flow to stop / start just to switch voices. I expected to be able to click a new voice and have it pick up on the next word / sentence with that voice. To compare voices I have to stop, click a new one, then start and wait for it to process. Then I can hear the new voice but its already been 15 seconds since I heard the last so what was the difference even?

Re: OpenAI Audio Models

#258

Post by dockerd »

setsewerd wrote: post #170
lukebuehler wrote: post #145

Earlier quoted context omitted.

Convert any file (pdf, epub, txt) to an audoibook, downloadable as mp3, or directly listenable via RSS feed in, say, Apple Potcasts app. Basically make one-off audiobooks for yourself or a few friends.

Any plans to make a Chrome extension variant? Been looking for a high quality and cheap TTS extension for ages (like ElevenLabs Human Reader, except with less absurd pricing)

What is your use-case here?

Re: OpenAI Audio Models

#260

Post by KeplerBoy »

solardev wrote: post #87
crazygringo wrote: post #75

This is astonishing. I can type anything I want into the "vibe" box and it does it for the given text. Accents, attitudes, personality types... I'm amazed. The level of intelligent "prosody" here -- the rhythm and intonation, the pauses and personality -- I wasn't expecting anything like this so soon. This is truly remarkable. It understands both the text and the prompt for how the speaker should sound. Like, we're g…

Guess that's why the video game voice actors are still on strike: https://en.m.wikipedia.org/wiki/2024%E2%80%93present_SAG-AFT... If we as developers are scared of AI taking our jobs, the voice actors have it much worse...

I don't see how a strike will do anything but accelerate the professions inevitable demise. Can anyone explain how this could ever end in favor of the human laborers striking?
Post reply on HN