The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
WaveNet: A Generative Model for Raw Audio
71–80 of 153 posts
Re: WaveNet: A Generative Model for Raw Audio
#72Re: WaveNet: A Generative Model for Raw Audio
#73https://en.wikipedia.org/wiki/ZRTP
At some point to authenticate both parties verify a short message by reading it to each other.
However, NSA has already tried to MitM that about 10 years ago by using voice synthesis. It was deemed inadequate at the time. Wonder if TTS improvements like these, change that game and make it more plausable scenario.
Re: WaveNet: A Generative Model for Raw Audio
#74So when I get the AI from one place, train it with the voices of hundreds of people from dozens of other sources, and then have it read a book from Project Gutenberg to an mp3... who owns the mechanical rights to that recording?
Every single person who had rights on the sources for audio you used. For the same reason, Google training neural networks with userdata is very legally doubtful – they changed the ToS, but also used data collected before the ToS change for that.
What if my 'AI' was a human who learned to speak by being trained with the voices of hundreds of people from dozens of other sources? What's the difference?
Those waters seem muddy. I think that'd be an interesting copyright case, don't think it's self evident.
Re: WaveNet: A Generative Model for Raw Audio
#75The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
It shot me forward to a time where people just click a button to generate music they want to listen to. If you really like the generation, you save it and share it. It wouldn't have all of the other aspects that we derive from human-produced music like soul/emotion (because we know it's coming from a human, not because of how it sounds), but it would be a cool application idea anyway.
Re: WaveNet: A Generative Model for Raw Audio
#76What's really intriguing is the part in their article where they explain the "babbling" of wavenet, when they train the network without the text input. That sounds just like a small kid imitating a foreign (or their own) language. My kids grow up bilingual and I hear them attempt something similar when they are really small. I guess it's like listening in to their neural network modelling the sound of the new languag…
Re: WaveNet: A Generative Model for Raw Audio
#77The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
If you train NNs at the phrase level and overfit, then you get something that is indeed more or less the same as cross-fading at random between short sections.
Piano music is very idiomatic, so you'll capture some typical piano gestures that way.
But I'd be surprised if the music stays listenable for long. Classical music has big structures, and there's a difference between recognising letters (notes), recognising phrases (short sentences), recognising paragraphs (phrase structures), and parsing an entire piece (a novel or short story with characters and multiple plot lines.)
Corpus methods don't work very well for non-trivial music, because there's surprisingly little consistency at the more complex levels.
NN synthesis could be an interesting thing though. If you trained an NN on $sounds$ at various pitches and velocity levels, you might be able to squeeze a large and complex collection of samples into a compressed data set.
Even if the output isn't very realistic, you'd still get something unusual and interesting.
Re: WaveNet: A Generative Model for Raw Audio
#78This is incredible. I'd be worried if I were a professional audiobook reader :)
While I don't doubt it'll be possible for a computer to match it with enough input data (both in voice and human adjustment), it'll probably be a while before we'll be there and when we are there it'll likely require a lot of adjustment on the part of a professional. A big part of narration is knowing when and where a part of the story requires additional voice acting (and understanding what is required). A machine generated narration would have to understand the story sufficiently to be able to do that correctly. They might be able to get the audio to sound as good as it would sound if I narrated it, but someone with talent in the area is going to be hard to match.
All of that aside, it's getting pretty close to "good enough". When it reaches that point, my hope is more books will have audio versions available[2] and in all likelihood, some books that would have been narrated by a person today will likely be narrated by technology when it reaches that point, limiting human narration only to the top x% of books.
[0] I always resell books or donate them. This book was so bad that the half-hour it took from my life felt like a tragedy. I threw it out to prevent someone from experiencing its awfulness -- even for free.
[1] I realized it was the same book at the point a story was told that I had only read in the first book (and found mildly humorous). The reason I hated the other book was that it was written in the first person as a New York cop. I couldn't form a mental picture and the character was entirely unbelievable and one dimensional. When narrated properly, that problem was eliminated.
[2] I "speed read" (not gimmicky ... scan/skimming) and consume a ton of text. I've been doing it for 20 years or so and find it difficult to read word-for-word as is required for enjoyment of fiction, so to "force" it, I stick with audio books for fiction and love them.
Re: WaveNet: A Generative Model for Raw Audio
#79What's really intriguing is the part in their article where they explain the "babbling" of wavenet, when they train the network without the text input. That sounds just like a small kid imitating a foreign (or their own) language. My kids grow up bilingual and I hear them attempt something similar when they are really small. I guess it's like listening in to their neural network modelling the sound of the new languag…
To my Australian English ears, the babbling sounded vaguely Scandinavian.
Re: WaveNet: A Generative Model for Raw Audio
#80This is incredible. I'd be worried if I were a professional audiobook reader :)
I'm an audiobook junkie and as far as professional narrators go, I think it'd be hard to replace a high-end performance with something computer generated and end up with the level of quality offered by the likes of a great narrator like Scott Brick. I mention him by name because it was him that made me realize how important good quality narration is. I had purchased a book at an airport bookstore on a whim and while…