Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

people.xiph.org

Post reply on HN
Print view View original

41–50 of 76 posts

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#41

Post by charlesism »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

    the music barely at all.
I suspect the reason that excerpt sounds so bad is because the music has several instruments playing at once. One doesn't generally design a vocoder to deal with more than one voice. As that except plays, you can hear that the most prominent instruments (eg: the bass at several moments) sound pleasing, albeit speech-like.

It would probably different from the original music, but pleasant, if one processed each track separately.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#42

Post by petercooper »

godelski wrote: post #39
petercooper wrote: post #31

Earlier quoted context omitted.

Here you go: http://no.gd/vega2.wav It holds up ridiculously well considering the entire song compresses down to 25392 bytes.

I'm getting a 404 on this

Curious, it definitely works, but the domain is "weird" enough that certain firewalls or proxies may have trouble, perhaps. I've put it at https://gofile.io/?c=F5gle3 as an alternative.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#43

Post by anisppp »

opportune wrote: post #35
gok wrote: post #8

Earlier quoted context omitted.

3 GFLOP/sec sounds like a lot but it's considerably less math than the radio DSPs inside any modern phone's baseband is doing during a phone call.

I don't know much about phone tech, are the basebands really doing math or just instrumenting? My assumption would be that there is just some sensor writing to a buffer at a high frequency but that whatever processes that buffer operates at a lower frequency.

Your question is hard to parse? What is instrumenting? If it helps though... the word “baseband” itself is the lower frequency containing just the bandwidth of the signal. Ie that is the lower frequency...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#44

Post by gok »

anisppp wrote: post #43
opportune wrote: post #35

Earlier quoted context omitted.

I don't know much about phone tech, are the basebands really doing math or just instrumenting? My assumption would be that there is just some sensor writing to a buffer at a high frequency but that whatever processes that buffer operates at a lower frequency.

Your question is hard to parse? What is instrumenting? If it helps though... the word “baseband” itself is the lower frequency containing just the bandwidth of the signal. Ie that is the lower frequency...

True but here I intended "baseband" to mean https://en.wikipedia.org/wiki/Baseband_processor

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#46

Post by klodolph »

Eli_P wrote: post #16
loudmax wrote: post #3

For those too impatient to read the details, check out the "Hear for yourself" examples toward the bottom of the page. They're reproducing decent sounding speech at 1.6 kbps. 1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-t…

I use --vbr --bitrate 16 and it feels indistinguishable from the original for podcasts. As opusenc takes only wav for input and does not use multiple cores, I had to write scripts for parallel re-encoding of stuff.

I like to use Makefiles for parallel encoding.

    OPUSFLAGS = --vbr --bitrate 16
    all: $(patsubst %.wav,%.opus,$(wildcard *.wav))
    .PHONY: all
    %.opus: %.wav
        opusenc $(OPUSFLAGS) $
Make -j4 or whatever. There are a few other ways to do this (e.g. xargs).

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#47

Post by sehugg »

petercooper wrote: post #15
jmvalin wrote: post #10

Earlier quoted context omitted.

Actually, this won't work at all for music because it makes fundamental assumptions that the signal is speech. For normal conversations, it should work, though for now the models are not yet as robust as I'd like (in case of noise and reverberation). That's next on the list of things to improve.

Here we go! This is the first minute or so of Penny Lane by The Beatles converted down to a 10KB .bin and then back to a .wav: http://no.gd/pennylane.wav .. unsurprisingly the vocals remain recognizable, but the music barely at all.

Sounds like a typical LPC encoder at a low bitrate, like maybe 5 kbps.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#48

Post by godelski »

petercooper wrote: post #42
godelski wrote: post #39

Earlier quoted context omitted.

I'm getting a 404 on this

Curious, it definitely works, but the domain is "weird" enough that certain firewalls or proxies may have trouble, perhaps. I've put it at https://gofile.io/?c=F5gle3 as an alternative.

That one works! And now I'm going to have nightmares.

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#49

Post by teraflop »

CamperBob2 wrote: post #27
Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

That was a fairly ignorant comment because the whole idea behind a speech codec is to compress and reproduce speech in a manner that allows one to at least recognize who's speaking. But I wonder if there isn't a gem in there somewhere. The essential expressive characteristics of a person's voice change much more slowly than the frame rate of any codec, and predictive coding alone doesn't cover all of the possibilitie…

This idea is the basis of an interesting plot point in Vernor Vinge's sci-fi novel A Fire Upon the Deep.

The book is set in a universe where long-distance, faster-than-light communication is possible, but extremely bandwidth-constrained. However, localized computational power is many orders of magnitude beyond what we have today. As a result, much of the communication on the "Known Net" is text (heavily inspired by Usenet) but you can also send an "evocation" of audio and/or video, which is extremely heavily compressed and relies on an intelligent system at the receiving end to reconstruct and extrapolate all the information that was stripped out.

The downside, of course, is that it can become difficult to tell which nuances were originally present, and which are confabulated.

Another example of the dangers of compression algorithms that are too clever for their own good: https://www.theregister.co.uk/2013/08/06/xerox_copier_flaw_m...

Re: A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet

#50

Post by lnanek2 »

CamperBob2 wrote: post #27
Causality1 wrote: post #20

Local text to speech quality blows this out of the water with a much smaller bandwidth footprint than 1.6kb/s. To me this sounds sort of like "I figured out how to eat a pinecone in less than three seconds". Impressive but not useful.

That was a fairly ignorant comment because the whole idea behind a speech codec is to compress and reproduce speech in a manner that allows one to at least recognize who's speaking. But I wonder if there isn't a gem in there somewhere. The essential expressive characteristics of a person's voice change much more slowly than the frame rate of any codec, and predictive coding alone doesn't cover all of the possibilitie…

Since this is implemented by a neural network, maybe all you have to do to obtain that is train it with different voices.
Post reply on HN