191–200 of 201 posts
Great stuff. Is your team interested in the STT problem?
Yes, we've started working on it and will have a range of stt models v soon. lmk if you have a prod use-case in mind?
Earlier quoted context omitted.
Right now it's outgoing calls for a small business client that checks information. Although if they call back they don't mind an automated system, on outgoing calls the person answering will often hang up if they detect AI right away, so we use a realistic custom voice with an accent. This is a mind numbing task that requires workers to make hundreds of calls each day with only minor variations, sometimes navigating…
I don't like the sound of that. Why do humans always need to spoil new advancements by finding the worst use cases?
The legitimate objection people have to AI in this use case is that it can be slow or stupid in a way that wastes time. By acting more humanlike, we signal that we are going to be closer to human level performance.
I'd love to see a monolingual Japanese model sometime in the future. Qwen3-tts works for Japanese in general, but from time to time it will mix with some Mandarin in between, making it unusable.
You could try a preprocessing step where you convert to hiragana, but I guess that would lose pitch accent information (e.g. 飴 vs 雨)
> - As a result,
> - When the string "明日["tomorrow"]" is entered into TTS, the TTS model [･皿･] outputs an ambiguous pronunciation that sounds like a mix of "asu" and "ashita" (something like "[asyeta]").
> From this, we found that by using the proposed method, it is possible to obtain data from private data in which the consistency between speech, graphemes, and phonemes is almost certainly maintained for more than 80% of the total.
> Another possible cause is a mismatch between the domain of the training data's audio (all [in read-aloud tones]) and the inference domain.
1. Sounds like general state of Japanese speech dataset is a mess
1.1. they don't maintain great useful correspondence between symbols to audio
1.2. they tend to contain too much of "transatlantic" voices and less casual speeches
2. Japanese speakers generally don't denote pronunciations for text
2.1. therefore web crawls might not contain enough information as to how they're actually pronounced
2.2. (potentially) there could be some texts that don't map to pronunciations
2.3. (potentially) maybe Japanese spoken and literal languages are still a bit divergent from each others
3. The situation for Chinese/Sinitic languages are likely __nowhere__ near as absurd, and so Chinese STT/TTS might not be well equipped to deal with this mess
4. This feels like much deeper mess than what commonly observed "a cloud in a sky" Japanese TTS problems such as obvious basic alignment errors(e.g. pronouncing "potatoes" as "tato chi")
0: https://xkcd.com/1425/
1: https://zenn.dev/parakeet_tech/articles/2591e71094ea58
2: https://qiita.com/maishikawa/items/dcadfeebf693080f0415It is based on onnx, so can i use with transformers.js and the browser?
Yes, someone already made a web demo for it: https://github.com/clowerweb/kitten-tts-web-demo (7 months ago). WebGPU support was marked experimental there, but transformer.js v4 (released last month) seems more stable now with some runtime/perf improvements: https://huggingface.co/blog/transformersjs-v4#performance--r...
I'm really curious: how does the inference speed of these <25MB models look on consumer GPUs? Also, are these models deterministic, or do they have a stochastic nature where you need to generate multiple takes to get the best prosody?
Earlier quoted context omitted.
shouldn't be hard. what backend/hardware are you interested in running this with? i'll add an example for using C++ onnx model. btw check out roadmap, our inference engine will be out 1-2 weeks and it is expected to be faster than onnx.
I want to run it in a website with Wasm and having the browser do the audio playback
Added kitten (nano only, for now, will move on to mini) to my "web tts thing": https://github.com/idle-intelligence/tts-web
To the folks and Kitten team: I'm working on TTS as a problem statement (for an application), and what is the best model at the latency/cost inference. I'm currently settling for gemini TTS, which allows for a lot of expressiveness, but a word at 150ms starts to hurt when the content is a few sentences. my current best approach is wrapping around gemini-flash native, and the model speaking the text i send it, which a…
I ran install instructions and it took 7.1GB of deps, tf you mean "tiny" ?