Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

people.xiph.org

Post reply on HN
Print view View original

51–60 of 76 posts

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#51

Post by jmvalin »

CamperBob2 wrote: post #27
Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

That was a fairly ignorant comment because the whole idea behind a speech codec is to compress and reproduce speech in a manner that allows one to at least recognize who's speaking. But I wonder if there isn't a gem in there somewhere. The essential expressive characteristics of a person's voice change much more slowly than the frame rate of any codec, and predictive coding alone doesn't cover all of the possibilitie…

The ceptrum that takes up most of the bits (or the LSPs in other codecs) is actually a model of the larynx -- another reason why it doesn't do well on music. Because of the accuracy needed to exactly represent the filter that the larynx makes, plus the fact that it can more relatively quickly, there's indeed a significant number of bits involved here.

The bitrate could definitely be reduced (possibly by 50%+) by using packets of 1 seconds along with entropy coding, but the resulting codec would not be very useful for voice communication. You want packets short enough to get decent latency and if you use RF, then VBR makes things a lot more complicated (and less robust).

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#52

Post by bravura »

jmvalin wrote: post #51
CamperBob2 wrote: post #27

Earlier quoted context omitted.

That was a fairly ignorant comment because the whole idea behind a speech codec is to compress and reproduce speech in a manner that allows one to at least recognize who's speaking. But I wonder if there isn't a gem in there somewhere. The essential expressive characteristics of a person's voice change much more slowly than the frame rate of any codec, and predictive coding alone doesn't cover all of the possibilitie…

The ceptrum that takes up most of the bits (or the LSPs in other codecs) is actually a model of the larynx -- another reason why it doesn't do well on music. Because of the accuracy needed to exactly represent the filter that the larynx makes, plus the fact that it can more relatively quickly, there's indeed a significant number of bits involved here. The bitrate could definitely be reduced (possibly by 50%+) by usin…

So can you do domain adaptation and get it to vocode my voice into Johnny Cash's? Larynx adaptation.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#53

Post by jmvalin »

bravura wrote: post #52
jmvalin wrote: post #51

Earlier quoted context omitted.

The ceptrum that takes up most of the bits (or the LSPs in other codecs) is actually a model of the larynx -- another reason why it doesn't do well on music. Because of the accuracy needed to exactly represent the filter that the larynx makes, plus the fact that it can more relatively quickly, there's indeed a significant number of bits involved here. The bitrate could definitely be reduced (possibly by 50%+) by usin…

So can you do domain adaptation and get it to vocode my voice into Johnny Cash's? Larynx adaptation.

In theory, it wouldn't be too hard to implement with an neural network. In theory. In practice, the problem is figuring out how to do the training because I don't have 2 hours of your voice saying the same thing as the target voice and with perfect alignment. I suspect it's still possible, but it's not a simple thing either.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#55

Post by qwerty456127 »

The examples sound excellent. Top (equal or better) of any text-to-speech synthesizer I've ever heard. I would love to start using it for audio books and for VoIP to save space and traffic as soon as possible. And a Linux-native text-to-speech synthesizer capable of producing speech of this quality is a thing I dream of (now the only option I know is booting to Windows and using Ivona voices)

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#56

Post by qwerty456127 »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

Hilariously nightmarish. I'm going to use this for my alarm clock...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#58

Post by cooper12 »

0-_-0 wrote: post #18

OK, so... Would it be possible to do something similar for video?

Similar as in the same approach, or as in "apply neural networks to all the things"? Because if it's the former, this approach was very specifically tailored to human speech, taking into account how much it can compress/interpolate qualities like pitch and the spectral envelope. That's far too specific to apply to video.

As for the latter, you'd have to perhaps feed Google Scholar the right incantations or ask someone with knowledge. As far as I know, video codecs already have a huge bag of tricks they use (for example the B-frames borrowed in this post). Even then, the key points in this codec were that firstly it's meant for use at very low bitrates, where existing codecs break down, and then secondly it's a vocoder, so it's converting audio to an intermediate form and resynthesizing it. That kind of lossiness is acceptable for audio, but I'm not sure how it would work acceptably for video.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#59

Post by Animats »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

Right. This form of compression assumes a primary single pitch, plus variations from that tone. You can hear it locking into different components of the song and losing almost everything else.

Heavy compression of voice is vulnerable to background noise.

I miss the classic telco 8K samples per second, 8 bits. We used to think that was crappy audio.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#60

Post by na85 »

zackbloom wrote: post #57

Just to put this in perspective, a traditional phone line encodes 56 Kb/s of data, which was believed to be the size channel to send the human voice with a reasonable quality. They are doing it in 1.6 Kb/s!

Aren't "traditional" aka POTS lines analog, and therefore not doing any encoding whatsoever?
Post reply on HN