Live data from Hacker News

Voicebox: Generative AI model for speech that generalizes across tasks

ai.facebook.com

111–120 of 128 posts

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#111
post #98

Earlier quoted context omitted.

I'm surprised you had such a negative reaction to the Hindi accent! To me, it was no more difficult to understand than my colleagues who speak English as a second language. To me, this is a style choice for the demo. Not evidence that they "fucked" it up. Accents are common - everyone has one! It's nice to see the model can support your personal voice even if it's not completely neutral English.

> It's nice to see the model can support your personal voice even if it's not completely neutral English There is no such thing as "neutral" English.

>Nonetheless, a form of speech known to linguists as General American is perceived by many Americans to be "accent-less", meaning a person who speaks in such a manner does not appear to be from anywhere in particular. The region of the United States that most resembles this is the central Midwest, specifically eastern Nebraska (including Omaha and Lincoln), southern and central Iowa (including Des Moines), parts of Missouri, Indiana, Ohio and western Illinois (including Peoria and the Quad Cities, but not the Chicago area).

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#112
post #90

Earlier quoted context omitted.

Cryptographic proof of personhood is going to be a thing, is it not? Outside of BigTech, Signal is as poised as WorldCoin to be just that.

I’m just not convinced that anything not tied directly to a government issued ID is going to be strong enough.

Most of the time you want to confirm that you're talking to someone from a given context -- they own a specific Twitter account, or you met them at a party last week, or they sent you an email or were present in a meeting that you want to have a conversation about.

Government ID doesn't help much with those -- it's actually the thing that is not strong enough.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#113
This is like qhwn Google published all those papers about their LLM tech, and ChatGPT just launched something that worked. It will end the same way for their voice tech if they never release it. Someone will release something just as good and take the market.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#114

Earlier quoted context omitted.

Caveat though: > if you can deliver same quality/better product for cheaper to customers, without requiring employees to be underpaid. This almost never happens. Cheaper? Yes. Same or better quality? Not a chance. Automated solutions tend to allow reducing quality way below what humans workers would want to do, or even could cheaply and reliably (i.e. doing worse job than a careless one takes actual effort/skill). Li…

I really wonder whether you have seen Midjourney, Stable Diffusion and ChatGPT or any other the recent trendy AI things. You can't find an illustrator who could "cheaply and reliably" do illustrations at Midjourney's level. You just can't. If you could you would have been the biggest contractor company in the world long time ago.

"Midjourney's level" is precisely the quality I'm talking about. It's impressive for what the computer can do, yes. It's not impressive when you find it coming out of a black box labeled "commercial commissioned art", not when compared to what used to come out of that box for about the same price. The images are... almost OK. But there's always something. A missing finger here, a tiny extra eye there, some psychedelic pattern faintly visible in the negative space, etc.

But what can you do? Every black box labeled "commercial commissioned art" is now returning similarly off images, almost but not quite there. They all dropped their prices a little, so there's that - while the few black boxes offering the quality that used to be normal now cost 2-3x of what used to be normal. Hard pass.

(Meanwhile, people operating the black boxes - i.e. companies or in-house departments churning out commercial graphics cheaply - are swimming in money made on firing all their minimum-wage artists, replacing them with Midjourney or SD, and pocketing the difference. Sure, they had to drop the prices a little to clear out remaining human-powered competitors, and they will have to drop them way further once the competition restarts in the earnest - but for a short moment, they all get to make small fortunes on selling shit output, that's 100+x cheaper to produce, at roughly the same price as mediocre one before.)

Can AI be used to generate much higher quality at the same cost as human art? Sure - you'll need to spend what you used to pay an artist, whom you just fired, on generating variants and a (cheaper, at least per unit of output) human select best ones - but yes, AI can give you much better quality for the same price. But AI can also give you same quality as before for cheaper, or somewhat worse quality for much cheaper. Which is the best option to choose?

The answer, I claim, is that there is no choice - competitive pressure will force everyone to go for shittiest quality the market can bear, sold almost at cost. This will satisfy enough demand that "standard quality" offering becomes something very expensive or outright unavailable, as economics of using minimum-wage factory artists suddenly stops working.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#115

Earlier quoted context omitted.

I'm not romanticizing quality of human work. I'm not claiming people give more shit than it seems. I'm claiming that with humans doing the work, quality can only get so low[0] - and automation lets you punch through that floor, achieving much lower quality standards. Or, put another way: > Removing humans from the loop often directly leads to improved quality. Yes, but improved quality for the same price means leavin…

> Case in point: automated checkout machines in stores. I don't know where you live, but I've never seen automated checkout machines. I only have seen self checkout machines. It requires the customer to do the cashier' job and that's all. The only reason it's not good is that it's not automated enough (if at all -- for me the self checkout machine is literally zero automation more than a regular cashier)

Yes, I meant self-checkout machines.

> The only reason it's not good is that it's not automated enough (if at all -- for me the self checkout machine is literally zero automation more than a regular cashier)

That's the point. But you are not the buyer of that automation, the store is. That automation displaced human cashiers and lowered the quality of service for customers, while generating better margins for the store (promptly eaten by competition). From your POV, i.e. customer's POV, it's not automated enough - but it's not going to be for quite a while, because there is no incentive to do it. The store doesn't stand to benefit much from additional automation, not enough to justify investment. Whether or not customers like it is irrelevant, as long as they're still coming in anyway.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#116
post #4

I think that the "star trek" use case of a live translation is super exciting. I think that this also will force people to have pass phrases that they use to authenticate phone calls with. I normally downplay when people bring up everyone signing everything with a public/private key (impractical for normal users) but clearly there will be a need for authentication protocols as AI proliferates

How "live" can translation ever really be? Properly translating anything from one language to another requires context.

Human translators do it in approximately real-time, see for example the United Nations. At most you might need to wait for the end of the sentence to translate if e.g. the verb/object ordering is different.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#117
post #24

> As with other powerful new AI innovations, we recognize that this technology brings the potential for misuse and unintended harm. In our paper, we detail how we built a highly effective classifier that can distinguish between authentic speech and audio generated with Voicebox to mitigate these possible future risks. We believe it is important to be open about our work so the research community can build on it and t…

From the paper:

> Model Transformer [Vaswani et al., 2017] with convolutional positional embedding [Baevski et al., 2020] and ALiBi self-attention bias [Press et al., 2021] are used for both the audio and the duration model. ALiBi bias for the flow step xt is set to 0. The audio model has 24 layers, 16 attention heads, 1024/4096 embedding/feed-forward network (FFN) dimension, 330M parameters. We add skip connections connecting symmetric layers (first layer to last layer, second layer to second-to-last layer, etc.) in the style of the UNet architecture. States are concatenated channel-wise and then combined using a linear layer. The duration model has 8 heads, 512/2048 embedding/FFN dimensions, with 8/10 layers for English/multilingual setup (28M/34M parameters in total). All models are trained in FP16.

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#119

Earlier quoted context omitted.

How does it compare to Voicebox in quality?

I would say that properly configured Tortoise is better, but that comes with the massive caveat that Tortoise: 1 - Is a real pain to get 'working right' - it's not even remotely batteries included and, more importantly: 2 - Is incredibly slow . I've been turning Heart Of Darkness into an audiobook as a unit test and it takes ~30m per paragraph, on average. Add to that the occasional hiccup where a block gets transcri…

FYI there’s also this fork for faster inference: https://github.com/152334H/tortoise-tts-fast

Re: Voicebox: Generative AI model for speech that generalizes across tasks

#120
post #54

Earlier quoted context omitted.

Not releasing code or weights under the false pretense of misuse.

False? The misuse opportunities are obvious.

Reproducing the code is a matter of time, and short at that.
Post reply on HN