Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

waxy.org

Post reply on HN
Print view View original

21–30 of 207 posts

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#21

Post by foobarian »

minimaxir wrote: post #10

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal mus…

It saddens me because of how much impact they had on my family as we played through the story line in Genshin and immersed in the world. At some point we met a few of the voice actors at a convention and they were like stars to us, while I'm sure their circumstances are as you describe.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#22

Post by sumtechguy »

dwringer wrote: post #8
mckirk wrote: post #4

My absolute favorite application of this tech so far is The Beach Boys singing 'Hurt'. It's the first time I seriously didn't notice any artifacts, and it just works so well even though it really shouldn't. Enjoy: https://youtu.be/gmNSFqyg_Z8

I don't know what I was expecting but that isn't Hurt, it's Surfin' USA with Hurt's lyrics that sound extremely jittery and grainy. I'm curious though if some AI soon could in fact synthesize the Beach Boys' style with the actual chords and melody from the NIN song, possibly with some of the pathos of Johnny Cash as well.

The one I found fun was the matrix ice ice baby mashup. That was sort of janky but good enough to be fun.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#23

Post by lhl »

satvikpendem wrote: post #9

What's the best open source text to speech? Eleven Labs and others are interesting but closed source. I want to use them mainly for audiobooks as I have a lot of ePubs and I'm just using the basic Google text to speech voices on my Android, via Moon+ Reader. It works fine but it's still more robotic than state of the art.

For neutral sounding very fast/efficient voices, I find Coqui TTS VITS models to be very good. For slower, more expressive voice or voice cloning I think the Coqui TTS XTTS is good (or you can look at the mrq/tortoise-tts).

I'm still awaiting a StyleTTS2 implementation. The audio samples sound top notch: https://styletts2.github.io/

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#24

Post by ticulatedspline »

satvikpendem wrote: post #9

What's the best open source text to speech? Eleven Labs and others are interesting but closed source. I want to use them mainly for audiobooks as I have a lot of ePubs and I'm just using the basic Google text to speech voices on my Android, via Moon+ Reader. It works fine but it's still more robotic than state of the art.

Bark seems pretty good

https://github.com/suno-ai/bark Demo at https://huggingface.co/spaces/suno/bark

In the couple samples I tried it was substantially better at picking up meaning compared to VALL-E-X

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#25

Post by pdntspa »

mecredis wrote: post #3

It's kind of wild that these tools just transfer a copy of these models every time they're spun up (whether it's to a Google Colab notebook or a local machine.) This must mean Hugging Face's bandwidth bill must be crazy, or am I missing something (maybe they have a peering agreement? heavily caching things?)

I really wish I could configure this crap to cache somewhere other than my C: drive

Or better yet, how about asking me where I want to store my models?

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#27

Post by darkerside »

dwringer wrote: post #8
mckirk wrote: post #4

My absolute favorite application of this tech so far is The Beach Boys singing 'Hurt'. It's the first time I seriously didn't notice any artifacts, and it just works so well even though it really shouldn't. Enjoy: https://youtu.be/gmNSFqyg_Z8

I don't know what I was expecting but that isn't Hurt, it's Surfin' USA with Hurt's lyrics that sound extremely jittery and grainy. I'm curious though if some AI soon could in fact synthesize the Beach Boys' style with the actual chords and melody from the NIN song, possibly with some of the pathos of Johnny Cash as well.

Yeah, I hate it to the point of being personally offended. It has nothing to do with Johnny Cash's rendition. I'd probably feel a bit better, but not much, if it were advertised as a NIN mashup.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#28

Post by zerojames »

minimaxir wrote: post #10

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal mus…

I am interested! You should write about what you find interesting; never worry if it will interest a particular group.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#29

Post by sumtechguy »

minimaxir wrote: post #10

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal mus…

What I find interesting is this aspect that eventually, these companies will hire some college kids who needs a couple thousand bucks and a free pizza. Have them read the right scripts. Sign the right 'give everything away' contract and just forever use their voice. Or do it sneaky. Have a voice assistant and in your ToS 'we can use a copy of your voice for anything'.

The existing voice actors will be just out of work. There will be a small cadre of groups that want real voice. But for some projects that will not be that important.

Its going to get crazy.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#30

Post by cm2012 »

darkerside wrote: post #27
dwringer wrote: post #8

Earlier quoted context omitted.

I don't know what I was expecting but that isn't Hurt, it's Surfin' USA with Hurt's lyrics that sound extremely jittery and grainy. I'm curious though if some AI soon could in fact synthesize the Beach Boys' style with the actual chords and melody from the NIN song, possibly with some of the pathos of Johnny Cash as well.

Yeah, I hate it to the point of being personally offended. It has nothing to do with Johnny Cash's rendition. I'd probably feel a bit better, but not much, if it were advertised as a NIN mashup.

It's definitely in the realm of "soulless"
Post reply on HN