Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

waxy.org

Post reply on HN
Print view View original

71–80 of 207 posts

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#71

Post by hiccuphippo »

sumtechguy wrote: post #29
minimaxir wrote: post #10

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal mus…

What I find interesting is this aspect that eventually, these companies will hire some college kids who needs a couple thousand bucks and a free pizza. Have them read the right scripts. Sign the right 'give everything away' contract and just forever use their voice. Or do it sneaky. Have a voice assistant and in your ToS 'we can use a copy of your voice for anything'. The existing voice actors will be just out of wor…

Mozilla has a voice data project where people already do it for free(dom) ;)

https://commonvoice.mozilla.org/en

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#72

Post by dylan604 »

minimaxir wrote: post #70
dylan604 wrote: post #66

Earlier quoted context omitted.

I pay little attention to SAG contracts, but after the Writer's Guild strike, I'd be expecting SAG to follow suit with major asks to protect its members from AI if they have not already covered it.

The current standard is the NAVA AI Rider: https://navavoices.org/2023/01/23/artificial-intelligence-ri... NAVA also has guidelines for protection against AI abuse: https://navavoices.org/synth-ai/

thanks. i have recently been asked by a couple of acquaintances that have done a few character voices in the past what I thought on AI and what can really be done with it. because of their infrequent performances, they aren't union members, but I'll pass along these links.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#73

Post by hinkley »

> Artifacts aside, it sounds like Michael Jackson doing a Weird Al impression?! Every line has a distinctly “white and nerdy” vibe: it loses any seriousness and edge, exaggerating words for comic effect and enunciating lyrics really clearly so the punchlines can be heard.

No, it sounds like someone doing doing an impression of Weird Al doing an impression of Michael Jackson. Someone whose mom told them they were special and they believed it.

These examples are standing on a ridge line, surveying the uncanny valley and looking for the best way to cross.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#74

Post by entrepy123 »

satvikpendem wrote: post #9

What's the best open source text to speech? Eleven Labs and others are interesting but closed source. I want to use them mainly for audiobooks as I have a lot of ePubs and I'm just using the basic Google text to speech voices on my Android, via Moon+ Reader. It works fine but it's still more robotic than state of the art.

POST-EDIT, CORRECTED ANSWER

I doubt it's currently actually "the best open source text to speech", but the answer I came up with when throwing a couple of hours at the problem some months ago was "ttsprech" [3].

Following the guide, it was pretty trivial to make the model render my sample text in about 100 English "voices" (many of which were similar to each other, and in varying quality). Sampling those, I got about 10 that were pretty "good". And maybe 6 that were the "best ones" (very natural, not annoying to listen to, actually sounded like a person by and large), and maybe 2 made the top (as in, a tossup for the most listenable, all factors considered).

IIRC, the license was free for noncommercial use only. I'm not sure exactly "how open source" they are, but it was simple to install the dependencies and write the basic Python to try it out; I had to write a for loop to try all the voices like I wanted. I ended using something else for the project for other reasons, but this could still be a fairly good backup option for some use cases, IMO.

PRE-EDIT, ERRONEOUS ANSWER

Same as above, but I had said "Silero" [0, 1, 2] originally, which I started trying out too, before switching to a third (less open) option.

  [0] https://github.com/snakers4/silero-models#text-to-speech
  [1] https://silero.ai
  [2] https://github.com/snakers4/silero-models#standalone-use
  [3] https://github.com/Grumbel/ttsprech#usage

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#75

Post by hinkley »

mckirk wrote: post #4

My absolute favorite application of this tech so far is The Beach Boys singing 'Hurt'. It's the first time I seriously didn't notice any artifacts, and it just works so well even though it really shouldn't. Enjoy: https://youtu.be/gmNSFqyg_Z8

The graininess of the recording covers over a lot of potential problems. But given that this attempt keeps the Beach Boy’s tempo and enunciation, I think this technique, whatever it is, would make a much more compelling version of Michael Jackson covering Eat It.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#76

Post by pixl97 »

supriyo-biswas wrote: post #16
minimaxir wrote: post #10

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal mus…

HN isn't the only community to write for. While most people here seem to be unsympathetic to such job concerns, unconventional articles do hit the front page from time to time. I'd like to read it, in any case.

The get rich at any cost type like to post on these articles at a higher rate I think. When you read a larger and broad range of HN posts you see a substantial part of the population here has concerns about this.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#77

Post by hinkley »

distantsounds wrote: post #18

The sampled voices sound neither like Michael Jackson nor Weird Al. A good effort, but a professional impersonator could likely do better on either front.

The best Michael Jackson interpreter in a town of 50,000 could do better than this. It’s… this is bad.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#78

Post by legitster »

dwringer wrote: post #8
mckirk wrote: post #4

My absolute favorite application of this tech so far is The Beach Boys singing 'Hurt'. It's the first time I seriously didn't notice any artifacts, and it just works so well even though it really shouldn't. Enjoy: https://youtu.be/gmNSFqyg_Z8

I don't know what I was expecting but that isn't Hurt, it's Surfin' USA with Hurt's lyrics that sound extremely jittery and grainy. I'm curious though if some AI soon could in fact synthesize the Beach Boys' style with the actual chords and melody from the NIN song, possibly with some of the pathos of Johnny Cash as well.

I agree. The "x words over y music" can be fun, but isn't really impressive as a true genre parody.

The one that always comes to mind for me is this video of an Eminem interview done from scratch as a Talking Heads song: https://www.youtube.com/watch?v=Kfl3N9nesRg

This is potentially something that generative AI could be good at doing (at least recreating vocals), but this parody of the Talking Heads required a lot of very clever insight into what made a good Talking Heads song and returned a convincing and novel melody. And I think we are still a ways off.

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#79

Post by [deleted] »

hinkley wrote: post #73

> Artifacts aside, it sounds like Michael Jackson doing a Weird Al impression?! Every line has a distinctly “white and nerdy” vibe: it loses any seriousness and edge, exaggerating words for comic effect and enunciating lyrics really clearly so the punchlines can be heard. No, it sounds like someone doing doing an impression of Weird Al doing an impression of Michael Jackson. Someone whose mom told them they were spec…

[deleted]

Re: Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning

#80

Post by NoMoreNicksLeft »

dylan604 wrote: post #39
NoMoreNicksLeft wrote: post #33

Earlier quoted context omitted.

We bought the $300/month plan for a few months earlier this year... and you'd only get 40 hours of audio generation for that. It wasn't really sufficient to our needs. How many audio books is 40 hours? Also, while its voice cloning was truly amazing, every once in awhile the voice would get a little nutty and sound like an insect just flew down their throat, or maybe they had an LSD flashback. Normal normal normal th…

>How many audio books is 40 hours? Are you reading War & Peace or Cat In The Hat?

I like to read with my eyes, not listen. I honestly have no idea how long an audio book is, hours-wise.

I've seen a few for download, and they're always like hundreds of meg, if not over a gig. And that's in mp3, where it should be compressed heavily.

Post reply on HN