Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

people.xiph.org

Post reply on HN
Print view View original

11–20 of 76 posts

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#11

Post by petercooper »

jmvalin wrote: post #10
loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

I tried it with music and the results were spooky. Very ethereal and ghostly. It was only with some classical music though, I might have to do a pop song next and share the results!

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#12

Post by jmvalin »

rasz wrote: post #5

3 Gflops, we are deep beyond diminishing returns here. Opus seems good enough.

Keep in mind that the very first CELP speech codec (in 1984) used to take 90 seconds to encode just 1 second of speech... on a Cray supercomputer. Ten years later, people had that running in their cell phones. It's not just that hardware keeps getting faster, but algorithms are also getting more efficient. LPCNet is already 1/100 the complexity of the original WaveNet (which is just 2 years old) and I'm pretty sure it's still far from optimal.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#13

Post by semi-extrinsic »

loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

For comparison, adaptive GSM encodings, which are in use for cellphones today, are also in the single-digit kbps.

https://en.wikipedia.org/wiki/Adaptive_Multi-Rate_audio_code...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#14

Post by nullc »

rasz wrote: post #5

3 Gflops, we are deep beyond diminishing returns here. Opus seems good enough.

Opus is awesome and covers a previously unmatched spectrum of use cases... but that isn't everything.

Opus isn't good enough to be a replacement for AMBE for use over radio. Opus doesn't make it easier to make very high quality speech synthesis, etc.

Opus loss robustness could be much better using tools from this toolbox-- and we're a long way from not wanting better performance in the face of packet loss.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#15

Post by petercooper »

jmvalin wrote: post #10
loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#16

Post by Eli_P »

loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

I use --vbr --bitrate 16 and it feels indistinguishable from the original for podcasts. As opusenc takes only wav for input and does not use multiple cores, I had to write scripts for parallel re-encoding of stuff.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#17

Post by bch »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

As imagined by Marilyn Manson...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#19

Post by petercooper »

bch wrote: post #17
petercooper wrote: post #15

Earlier quoted context omitted.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

As imagined by Marilyn Manson...

Pretty much! It shows off how the codec works to a great extent though as it seems to be misinterpreting parts of the music to be the pitch of the speech, so Paul's voice sounds weird at the start of most lines but okay throughout the lines.

I've also run a BBC news report through the program with better results although it demonstrates that any background noise at all can throw things off significantly: https://twitter.com/peterc/status/1111736029558517760 .. so at this low bitrate, it really is only good for plain speech without any other noise.

Post reply on HN