Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

people.xiph.org

Post reply on HN
Print view View original

1–10 of 76 posts

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#2

Post by mmastrac »

I've been waiting for someone to do this* with audio and/or video. Amazing work.

Also worth reading this related link: https://www.rowetel.com/?p=6639

There's some really exciting progress that could be made in this space. The page mentions that they could use it as a post-processing filter for Opus for better decoding to avoid changing the bitstream. It could also be useful as a way to accommodate for packet loss and recover "just enough" to avoid interrupting the conversation.

* encoding audio through a neural network for network transmission

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#3

Post by loudmax »

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps.

1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-to-text, then text-to-speech.

The original recordings are a little stiff, and the encoded speech is a little more stiff. It isn't perfect, but it's decent. It'll be interesting to hear this technique applied to normal conversation. If regular speech holds up as well as their samples, it should be perfectly adequate for conversational speech. At 1.6 kbps, which is absurd.

Also, I wonder how well this technique could be applied to music. My guess is that it won't do justice to great musicians ... but it might be good enough simple pop tunes.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#4

Post by dietrichepp »

loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

I’ve done experiments with Opus that produce intelligible (but ugly) speech at 2.3 kbit/s. It involves downsampling the Opus stream at the packet level—e.g. transmit only one out of every three packets. It was surprisingly easy. Nothing as sophisticated as what’s going on here.

Also based on work by Xiph. Possibly using the same LPC used here.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#10

Post by jmvalin »

loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.
Post reply on HN