Does it sound fairly robotic/static-y to anyone else or just me? Doesn't sound any better than any other TTS software I've tried and in fact sounds a bit worse, like it's noisy.
Bark – Text-prompted generative audio model
61–70 of 99 posts
Re: Bark – Text-prompted generative audio model
#62Man I know this is HN, and I know we have a certain decorum we should be maintaining, but with the recent activity in this field the most appropriate response to these posts is "4bit when?" or "f16 when?". Not sure which one is applicable. I am having no luck running it on a 6GB vram gpu, so I guess its the 16 bit floating point one.
I'm successfully running it on a 12GB GPU (while it downloads some 12.1GB of model data on first run, the highest GPU memory usage was ~6.5GB, settling back down to around 5GB), however the results are nothing like the samples given on the github page. Using the exact code given and in the runs I've tried the results are rather terrible.
I'm not being negative -- some of the samples are really neat on their page -- and I know there is some idiosyncrasy of my setup that is causing issues, though it is a pretty typical conda + pytorch with CUDA 11.8.
Playing with the text and waveform temp from their defaults 0.7 is yielding some semi-decent results, but it feels essentially random.
Re: Bark – Text-prompted generative audio model
#63Hello how are you doing
Re: Bark – Text-prompted generative audio model
#64Am I hallucinating or didn't several of the examples have background audio artifacts, like it's been trained on speech with noisy backgrounds, I'm guessing audio from movies paired with subtitles? Having random background audio can make it quite hard to use in production.
>Am I hallucinating or didn't several of the examples have background audio artifacts, like it's been trained on speech with noisy backgrounds, I'm guessing audio from movies paired with subtitles? Having random background audio can make it quite hard to use in production.
The other side of that problem is an opportunity. That's why the same model can also generate music, background noise and sound effects. And it's just because the prompt specifies those things explicitly. The input is truly semantic, so the output is rich and reflects that context. Is your input text sounds like it came from a speech, then there's a high chance your output audio will sound like a megaphone in a public space with crowd reactions and maybe even applause.
Re: Bark – Text-prompted generative audio model
#65Hey, one of the Suno founders/creators of Bark here. Thanks for all the comments, we love seeing how we can improve things in the future. At Suno we work on audio foundation models, creating speech, music, sounds effects etc…. Text to speech was a natural playground for us to share with the community and get some feedback. Given that this model is a full GPT model, the text input is merely a guidance and the model ca…
I like the emphasis tags, it's something that is not seen with a lot of these transformer models.... things like [laughs] makes alot of sense... i could see where hundreds or possibly thousands of emphasis style tags could be added to support a vast array of intonations in human speech. i.e. [yells], [shouts], [cries], [crying], [whispers], [sarcasasm], etc
Re: Bark – Text-prompted generative audio model
#66Hey, one of the Suno founders/creators of Bark here. Thanks for all the comments, we love seeing how we can improve things in the future. At Suno we work on audio foundation models, creating speech, music, sounds effects etc…. Text to speech was a natural playground for us to share with the community and get some feedback. Given that this model is a full GPT model, the text input is merely a guidance and the model ca…
This tech will be used by crooks to automate attacks. Generate the language using GPT-4 and the audio using Bark, and then start making phone calls. Because it’s open source, all you need is GPUs. This is not a criticism. I’m impressed and grateful for the openness. Everyone needs to wake up and recognize that these attacks are coming at us essentially right now.
Recommend to encode sub-audible tracking symbols in synthesize speech patterns that includes GPS, IP, timestamp, country of origin. We do that already in bootleg movies so we can apply similar methods in synthesizes speech.
Re: Bark – Text-prompted generative audio model
#67Earlier quoted context omitted.
Amazing work so far! Do you have any sense about how difficult it would be to enable M1/M2 or CoreML support?
thanks, the model itself is a pretty vanilla gpt model based heavily on karpathy's nanogpt, so should not need too many bells and whistles to get it running on specific architectures. that said i have very little experience with platform specific development, so would looove some help from the community :)
Could you stick a torch.compile in the inference and training code, maybe gated behind a flag? This should help AMD/Nvidia performance (and probably other vendors soon) significantly.
PyTorch themselves used nanoGPT training as demo for this: https://pytorch.org/blog/accelerating-large-language-models/
Re: Bark – Text-prompted generative audio model
#68Earlier quoted context omitted.
This tech will be used by crooks to automate attacks. Generate the language using GPT-4 and the audio using Bark, and then start making phone calls. Because it’s open source, all you need is GPUs. This is not a criticism. I’m impressed and grateful for the openness. Everyone needs to wake up and recognize that these attacks are coming at us essentially right now.
Recommend to encode sub-audible tracking symbols in synthesize speech patterns that includes GPS, IP, timestamp, country of origin. We do that already in bootleg movies so we can apply similar methods in synthesizes speech.
It's open source, and even if it wasn't, it would likely be pretty easy to remove those (not that you would get GPS) from an audio file.
Re: Bark – Text-prompted generative audio model
#69Very cool. Side note: bark-gpt.com is already taken for a dog translator: "The world’s first AI powered, real-time communications tool between humans and their furry best friends."[0] I only know this because my law firm partner's name is Bark, and I wanted to automate some legal work and name the software "Bark GPT" after him. [0] https://www.bark-gpt.com/
I want to see the training set for this
Honestly you need to annotate datasets for barks and we're building this up with our users. Otherwise it's unclear what's happening even with video (we tried to build a YouTube/Instagram scraper and it was very unreliable even with vet tech annotations).
Re: Bark – Text-prompted generative audio model
#70“Last nights Phish performance is proof that God loves us.” But in Portuguese.
https://suno-ai.notion.site/Bark-Examples-5edae8b02a604b54a4...