Seems like Gboard is incompatible with it. Is there a good enough open source alternative to Gboard in 2024 that has smooth glide-typing and a similar layout?
Transcribro: On-device Accurate Speech-to-text
21–30 of 66 posts
Re: Transcribro: On-device Accurate Speech-to-text
#22Earlier quoted context omitted.
Considering it uses Whisper, it's probably not streaming
I did some core work on TTS at Google, at several layers, and I've never quite understood what people mean by streaming vs. not. In each and every case I'm familiar with, streaming means "send the whole audio thus far to the inference engine, inference it, and send back the transcription" I have a Flutter library that does the same flow as this (though via ONNX, so I can cover all platforms), and Whisper + Silero is…
Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things:
- Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the whole system, for instance an encoder can run in this fashion along audio as it comes in even if the final step of the system then runs in a non-streaming fashion.
- Produce partial results while the user is speaking: This can be just a UI nice to have, but it can also be much deeper, eg, a system can be activating on words or phrases in the input before the user is finished speaking which can dramatically change latency.
- Better segmentation: Whisper + Silero is just using VAD to make segments for Whisper, this is not at all the best you can do if you are actually decoding while you go. Looking at the results as you go allow you to make much better and faster segmentation decisions.
Re: Transcribro: On-device Accurate Speech-to-text
#23This is cool, I get to read another Jetpack Compose codebase since I am halfway through migrating our app to Jetpack. So this helps a lot.
Re: Transcribro: On-device Accurate Speech-to-text
#24Earlier quoted context omitted.
I think in practical terms (at least for me): - streaming == I talk and the text appears as I talk - batched == I talk, and after I'm done talking some processing happens and the text gets populated
Gotcha, then, it's "not even wrong" in the Pauli sense to say Whisper isn't streaming
It is not streaming in the way people normally use this term. It's a fuzzy notion but typically streaming means something encompassing:
- Processing and emitting results on something closer to word by word level - Allowing partial results while the user is still speaking and mid-segment - Not relying on an external segmenter to determine the chunking (and therefore also latency) of the output.
Re: Transcribro: On-device Accurate Speech-to-text
#25Earlier quoted context omitted.
I did some core work on TTS at Google, at several layers, and I've never quite understood what people mean by streaming vs. not. In each and every case I'm familiar with, streaming means "send the whole audio thus far to the inference engine, inference it, and send back the transcription" I have a Flutter library that does the same flow as this (though via ONNX, so I can cover all platforms), and Whisper + Silero is…
Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things: - Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the…
The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally.
Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative chipset vendor for that (read: wasn't possible even at scale, carrots, presumably sticks, and with nuch cajoling. So it works on iPhones consistently.)
The partial results are obtained by running inference on the entire audio so far, and silence is determined by VAD, on every stack I've seen that is described as streaming
I find it hard to believe that Google and Apple specifically, and every other audio stack I've seen, are choosing to do "not the best they can at all"
Re: Transcribro: On-device Accurate Speech-to-text
#26Earlier quoted context omitted.
Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things: - Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the…
The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally. Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative c…
This is exactly what Google ASR does. Give it a try and watch how the results flow back to you, it certainly is not waiting for VAD segment breaking. I should know.
Streaming used to be something people cared about more. VAD is always part of those systems as well, you want to use it to start segments and to hard cut-off, but it is just the starting off point. It's kind of a big gap (to me) that's missing in available models since Whisper came out, partly I think because it does add to the complexity of using the model, and latency has to be tuned/traded-off with quality.
Re: Transcribro: On-device Accurate Speech-to-text
#27Seems like Gboard is incompatible with it. Is there a good enough open source alternative to Gboard in 2024 that has smooth glide-typing and a similar layout?
Any of these should work.
https://github.com/Helium314/HeliBoard
https://github.com/openboard-team/openboard
https://github.com/rkkr/simple-keyboard (guessing, since AOSP Keyboard works and this is a fork)
Not open source: https://www.microsoft.com/en-us/swiftkey
Does not have glide/swipe (reserved for symbols), but I just installed and giving it a shot: https://github.com/Julow/Unexpected-Keyboard
Re: Transcribro: On-device Accurate Speech-to-text
#28Looks similar to the new FUTO keyboard: https://voiceinput.futo.org/
Anything like that available for iOS?
iOS already has on-device dictation built into the standard keyboard.
Years ago it got sent to the cloud, but as long as you have an iPhone from the past few years it's on-device.
Re: Transcribro: On-device Accurate Speech-to-text
#29Re: Transcribro: On-device Accurate Speech-to-text
#30Earlier quoted context omitted.
Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things: - Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the…
The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally. Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative c…
This is a complete non sequitur lol. FYI whisper is not a streaming model though it can, with some work, be adapted to be one.