Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Voice: Real-Time Neural Text-To-Speech

research.baidu.com

Post reply on HN
Print view View original

31–40 of 80 posts

Re: Deep Voice: Real-Time Neural Text-To-Speech

#31

Post by stevenh »

mbrookes wrote: post #21
mrmaximus wrote: post #2

Interesting. They are not TTS like we are accustomed to, they are replicating a specific persons voice with TTS. Listen to the ground-truth recordings at the bottom and then the synthesized versions above. "Fake News" is about to get a lot more compelling when you can make anyone say anything as long as you have some previous recordings of their voice.

> "Fake News" is about to get a lot more compelling hen you can make anyone say anything as long as you have some previous recordings of their voice. Adobe has already developed that technology: https://arstechnica.co.uk/information-technology/2016/11/ado... Now imagine combining it with this: Face2Face: Real-time Face Capture and Reenactment of RGB Videos https://www.youtube.com/watch?v=ohmajJTcpNk Perhaps using the…

I agree and I've upvoted you, but I feel it's worth pointing out that Adobe's claim about their own progress in this field was fake news.

https://www.youtube.com/watch?v=I3l4XLZ59iw&t=2m34s

"Wife" sounds exactly the same in both places. All they did was copy the exact waveform from one point to another. Nothing is being synthesized.

https://www.youtube.com/watch?v=I3l4XLZ59iw&t=3m54s

The word "Jordan" is not being synthesized. The speaker was recorded saying "Jordan" beforehand for this insertion demo and they're trying to play it off as though it was synthesized on the fly. This is a scripted performance and Jordan is feigning surprise.

https://www.youtube.com/watch?v=I3l4XLZ59iw&t=4m40s

The phrase "three times" here was prerecorded.

This was a phony demonstration of a nonexistent product. Reporters parroted the claims and none questioned what they witnessed. Adobe falsely took credit and received endless free publicity for a breakthrough they had no hand in by staging this fake demo right on the heels of the genuine interest generated by Google WaveNet. I suppose they're hoping they'll have a real product ready by whatever deadline they've set for themselves.

To be clear, I like Adobe and I think it's a cunning move on their part.

Re: Deep Voice: Real-Time Neural Text-To-Speech

#32

Post by mbrookes »

stevenh wrote: post #31
mbrookes wrote: post #21

Earlier quoted context omitted.

> "Fake News" is about to get a lot more compelling hen you can make anyone say anything as long as you have some previous recordings of their voice. Adobe has already developed that technology: https://arstechnica.co.uk/information-technology/2016/11/ado... Now imagine combining it with this: Face2Face: Real-time Face Capture and Reenactment of RGB Videos https://www.youtube.com/watch?v=ohmajJTcpNk Perhaps using the…

I agree and I've upvoted you, but I feel it's worth pointing out that Adobe's claim about their own progress in this field was fake news. https://www.youtube.com/watch?v=I3l4XLZ59iw&t=2m34s "Wife" sounds exactly the same in both places. All they did was copy the exact waveform from one point to another. Nothing is being synthesized. https://www.youtube.com/watch?v=I3l4XLZ59iw&t=3m54s The word "Jordan" is not being sy…

Thanks for the detailed breakdown. The irony is not lost!

Re: Deep Voice: Real-Time Neural Text-To-Speech

#33

Post by shanxS »

dresaj8 wrote: post #13
shanxS wrote: post #11

Earlier quoted context omitted.

Lex by AWS. Its the same deep learning tech. used as used by Alexa

does lex actually translate speech to text for you? i was under the impression that it was for conversational bots.

Yea.. you are right.. I just assumed that would fit any use case where speech to text is needed. Which clearly is wrong.

Re: Deep Voice: Real-Time Neural Text-To-Speech

#36

Post by deepnotderp »

PieSquared wrote: post #16

Hey there! I'm one of the authors of the paper and I'm happy to answer any questions anyone may have! Make sure to check out the paper on arxiv as well.

Hey, one quick question, did the QRNNs work better and faster than LSTMs out of the box, or did you guys have to tune hyperparameters?

Re: Deep Voice: Real-Time Neural Text-To-Speech

#37

Post by PieSquared »

deepnotderp wrote: post #36
PieSquared wrote: post #16

Hey there! I'm one of the authors of the paper and I'm happy to answer any questions anyone may have! Make sure to check out the paper on arxiv as well.

Hey, one quick question, did the QRNNs work better and faster than LSTMs out of the box, or did you guys have to tune hyperparameters?

We didn't actually try LSTMs, because we train in 1.25 second chunks, so running an LSTM for several hundred timesteps would drastically slow down training. Our per iteration time was in the 200-500 milliseconds, and using an LSTM or GRU would likely bump that into the 1-3 second range, maybe more, whereas the QRNN conditioning actually make it cheaper than the transposed convolution conditioning by 20-40%.

The upsampling procedure is quite finicky, so we had quite a few iterations there, but we didn't have to tune hyperparameters too much of the QRNN itself. Once we implemented the QRNN in CUDA for TensorFlow and got it to train, it worked without too much trouble.

Our collaborators in Beijing mentioned that bidirectional LSTMs also worked in a similar way, though.

Re: Deep Voice: Real-Time Neural Text-To-Speech

#38

Post by PieSquared »

bmc7505 wrote: post #29
PieSquared wrote: post #16

Hey there! I'm one of the authors of the paper and I'm happy to answer any questions anyone may have! Make sure to check out the paper on arxiv as well.

Hi Andrew, congratulations on your result! A few questions, feel free to answer one or any. How close do you think you are to having fully end-to-end models for speech? Are you optimistic we can get speech synthesis to run on mobile devices in the near future? Do the inference optimizations (particularly sample embedding and layer inference) generalize well to other architectures, like speech recognition? It seems th…

Thank you!

For fully end-to-end models, it's hard to say exactly. The Char2Wav paper demonstrates that there is hypothetically an architecture and a set of weights that can do synthesis end-to-end, but we cannot yet train such a system. On Reddit, one of the Char2Wav authors comments that they tried training it directly and didn't get great results, and at SVAIL we've also had some trouble doing so. I think it is very likely going to happen in the next several months or year, but we don't yet know exactly what needs to happen in order to get it to work.

As for inference, some of the inference optimizations do generalize. In fact, the GPU optimizations (persistent kernels) were originally developed by our systems team, and published in the Persistent RNN [0] paper. (This is a really powerful technique that CUDA makes very hard to implement, and I have a massive amount of respect for the folks who managed to make it work!) Persistent RNNs make training at close-to-peak-FLOPs with very low batch sizes plausible, and make GPU WaveNet inference plausible. At the moment, our CPU kernels are much more promising, but we don't know whether that will stay the case. For mobile, I think it is possible to get the current systems to work on fairly powerful mobile CPUs with a bunch more work into optimization and low-level assembly, but we haven't done it yet so time will tell.

[0] https://svail.github.io/persistent_rnns/ and http://jmlr.org/proceedings/papers/v48/diamos16.pdf

Re: Deep Voice: Real-Time Neural Text-To-Speech

#40

Post by PieSquared »

modeless wrote: post #20
PieSquared wrote: post #18

Earlier quoted context omitted.

We are not currently releasing any code, but hopefully the paper on arxiv is enough to make it easy to reproduce the result. We use TensorFlow for writing and training the model and c++ with a lot of hand optimizations for inference, with assembly kernels written with PeachPy (which is an awesome piece of software!)

I didn't know Baidu uses TensorFlow! Is that the main framework there now or are others in use?

Baidu as a company doesn't use TensorFlow (as far as I know). We have our own high-performance and easy to use open source framework called PaddlePaddle [0], which is quite powerful and flexible.

However, the Baidu Research Silicon Valley AI Lab (SVAIL) allows researchers and research teams to use whatever frameworks they want to, and we have projects using TensorFlow, Torch, our own SVAIL-internal internal high-performance RNN framework, and PaddlePaddle. Using our own framework sometimes allows us to work on very high-performance implementations of various primitives and techniques that would be harder to do without complete control over the source code.

[0] http://www.paddlepaddle.org/

Post reply on HN