Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

StreamVC: Real-Time Low-Latency Voice Conversion

research.google

Post reply on HN
Print view View original

1–10 of 44 posts

Re: StreamVC: Real-Time Low-Latency Voice Conversion

#3

Post by gnat »

From the poster:

In this work, we propose a light-weight (~20M param.) causal voice conversion solution that can run in real-time with low latency on a commercially available mobile device. The key design elements are: (1) using a causal encoder to learn soft speech units; (2) injecting whitened f0 to improve pitch stability without leaking source speaker info.

In our later V2 version, we found that f0 rescaling followed by a NSF-style harmonic-plus-noise conditioning (as is done in RVC) results in better quality.

Re: StreamVC: Real-Time Low-Latency Voice Conversion

#4

Post by [deleted] »

gnat wrote: post #3

From the poster: In this work, we propose a light-weight (~20M param.) causal voice conversion solution that can run in real-time with low latency on a commercially available mobile device. The key design elements are: (1) using a causal encoder to learn soft speech units; (2) injecting whitened f0 to improve pitch stability without leaking source speaker info. In our later V2 version, we found that f0 rescaling foll…

[deleted]

Re: StreamVC: Real-Time Low-Latency Voice Conversion

#6

Post by modeless »

huac wrote: post #5

The samples were released a while back: https://google-research.github.io/seanet/stream_vc/

Not a very good demo page. It's difficult to judge real world quality with such unenthusiastic reading, unrealistic sentences, and unfamiliar voices. Typical of speech papers. It would be much better if celebrities were used as target voices, as we all know what they sound like and can therefore judge quality better. But I suppose that would be too controversial for Google.

In general I think it is silly that voice cloning research has focused so much (exclusively?) on cloning voices from just a few seconds of audio. It puts a pretty low ceiling on quality. Many nuances of a person's communication style will not be contained in such a small amount of data. Sure you can match their pitch and timbre, but voice cloning should be more than that.

Post reply on HN