Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

people.xiph.org

Post reply on HN
Print view View original

21–30 of 76 posts

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#21

Post by jmvalin »

petercooper wrote: post #19
bch wrote: post #17

Earlier quoted context omitted.

As imagined by Marilyn Manson...

Pretty much! It shows off how the codec works to a great extent though as it seems to be misinterpreting parts of the music to be the pitch of the speech, so Paul's voice sounds weird at the start of most lines but okay throughout the lines. I've also run a BBC news report through the program with better results although it demonstrates that any background noise at all can throw things off significantly: https://twit…

Well, in the case of music, what happens is that due to the low bit-rate there are many different signals that can produce the same features. The LPCNet model is trained to reproduce whatever is the most likely to be a single person speaking. The more advanced the model, the more speech-like the music is likely to turn

When it comes to noisy speech, it should be possible to improve things by actually training on noisy speech (the current model is trained only on clean speech). Stay tuned :-)

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#22

Post by walrus01 »

For those interested in low bandwidth audio codecs, take a look at the voice codec used for Iridium handheld satellite phones, which was finalized in about 1998. Fully twenty plus years ago.

It doesn't sound the best, but consider the processing power constraints it was designed with...

https://en.wikipedia.org/wiki/Iridium_Communications#Voice_a...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#23

Post by est31 »

Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

The difference is that it uses the voice of the original person. And STT is rather erroneous. The WER is in the single percent range :).

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#24

Post by kstrauser »

Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

You really can't think of any uses for being able to push 40 simultaneous speech connections through a single 64Kbps voice channel?

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#26

Post by jmvalin »

walrus01 wrote: post #22

For those interested in low bandwidth audio codecs, take a look at the voice codec used for Iridium handheld satellite phones, which was finalized in about 1998. Fully twenty plus years ago. It doesn't sound the best, but consider the processing power constraints it was designed with... https://en.wikipedia.org/wiki/Iridium_Communications#Voice_a...

Iridium appears to be using a vocoder called AMBE. Its quality is similar to the one of the MELP codec from the demo and it also runs at 2.4 kb/s. LPCNet at 1.6 is a significant improvement over that -- if you can afford the complexity of course (at least it'll work on a phone now).

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#27

Post by CamperBob2 »

Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

That was a fairly ignorant comment because the whole idea behind a speech codec is to compress and reproduce speech in a manner that allows one to at least recognize who's speaking.

But I wonder if there isn't a gem in there somewhere. The essential expressive characteristics of a person's voice change much more slowly than the frame rate of any codec, and predictive coding alone doesn't cover all of the possibilities. There are also not that many unique human voices in the world. If you had several thousand people read a given passage and asked me to listen to them and tell me which voice belongs to a close friend or relative, I doubt I could do it.

So, if a codec could adequately pigeonhole the speaker's inflection, accent, timbre, pacing, and other characteristics and send that information only once per transmission, or whenever it actually changes, then a text-to-speech solution with appropriate metadata describing how to 'render' the voice might work really well.

Put another way, I doubt the nerves between your brain and your larynx carry anything close to 1.6 kbps of bandwidth. Near-optimal compression might be achieved by modeling the larynx accurately alongside the actual nerve signals that drive it, rather than by trying to represent both with a traditional frame-based predictive model.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#29

Post by walrus01 »

jmvalin wrote: post #26
walrus01 wrote: post #22

For those interested in low bandwidth audio codecs, take a look at the voice codec used for Iridium handheld satellite phones, which was finalized in about 1998. Fully twenty plus years ago. It doesn't sound the best, but consider the processing power constraints it was designed with... https://en.wikipedia.org/wiki/Iridium_Communications#Voice_a...

Iridium appears to be using a vocoder called AMBE. Its quality is similar to the one of the MELP codec from the demo and it also runs at 2.4 kb/s. LPCNet at 1.6 is a significant improvement over that -- if you can afford the complexity of course (at least it'll work on a phone now).

Based on my previous experience with Iridium I believe it actually operates at a data rate up to about 3 to 3.2 kb/s. 2400 bps of it is actual usable voice payload, the remaining 600 bps is FEC.

Iridium data (not the new next-generation network) service is around the same speed, it's 2400 bps + whatever compression v42bis can gain you. For plain text and stuff it can be a bit faster, something that's already incompressible by v42bis will be right around 2400 baud.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#30

Post by bonzini »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

Can you try it with Tom's Diner by Suzanne Vega? It's sung without any instruments, and an early version of MP3 reportedly was a disaster on that song.
Post reply on HN