Bark: A transformer based text to audio system

Incredible. To get that level of quality and realism and character in the voices, and to be able to clone the damn thing and play with it. What a time!

I'll link my Bark fork with long audio generation and other features on the root thread, I suppose: https://github.com/JonathanFly/bark

There's going to be a big update this week with some new stuff I haven't talked about. And a bunch of amazing, clear voices, with a huge variety of styles, that blow the default Suno voices out of the water. Arguably even better than Eleven in some ways. I'm excited even though I have nothing to DO with the voices!

Don't get too attached though. I was just playing around and made a Bark fork and it got more popular than expected. And now I'm dreading a future full of hours of unpaid support and maintenance that I definitely can NOT afford, for a software product I don't even really have a personal use case for. I'm not generating my own audiobooks or anything, I won’t be using it long term myself, I was just curious what Bark could do. (Turns out a LOT more than you might think at first glance, as you'll see this week.) So I'm already trying to work out how I can elegantly wind this thing down and transition people somewhere else. But I'll keep it updated for at least a little while.

I've been following Bark and its forks looking for a stable working cloning solution (for now, straight approaches at generating and using the npz files does not work), and oh boy. I have never seen so much low-quality git activity like in these repos. It really got me worried.

Yeah I was trying to figure out how good it was in Korean. The cadence and flow was pretty good but there was kind of artifacts in the audio. Then I check the samples of the default audio prompts for Korean any my god, they were godawful. Switching it up made a world of difference.

>Yeah I was trying to figure out how good it was in Korean. The cadence and flow was pretty good but there was kind of artifacts in the audio. Then I check the samples of the default audio prompts for Korean any my god, they were godawful. Switching it up made a world of difference.

I have a decent amount really clear Korean voices BTW. That and French, from people asking on Discord. But I can't judge the accents only that they are clear speakers.

Korean was interestingly the lone somewhat-coherent one-shot long term music sample I ever managed to get out of Bark.

https://www.youtube.com/watch?v=4pV9d25KqCE

The second music bit in this Youtube was one continuous generation where the last prompt was used as the history for the next, with no cherry picking or assembling the clips, just one solid segment. And it sort of holds together for like almost a minute!

I was excited because I thought maybe Bark could be like a real-time OpenAI Jukebox. But that was literally the only time so far where using a full-feedback held together like that. You can kind of 'cheat' it to by using a very popular song as the input text, and sometimes Bark will produce the appropriate melody. But of course that's not really the point of using your own text. I have some ideas for making it more coherent, but nothing easy. Too bad, Jukebox is just SO SLOW.

Actually with what I know now I should re-render this and clean up the distortion. At the time I couldn't do it. Though I only have the first segment prompt.

I have a decent amount really clear Korean voices BTW. But I can't judge the accents, only that they are clear.