Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transcribro: On-device Accurate Speech-to-text

github.com

Post reply on HN
Print view View original

31–40 of 66 posts

Re: Transcribro: On-device Accurate Speech-to-text

#31

Post by r2_pilot »

refulgentis wrote: post #25
opprobium wrote: post #22

Earlier quoted context omitted.

Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things: - Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the…

The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally. Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative c…

Thank you for your insight. It confirms some of my suspicions working in this area (you wouldn't happen to know anybody who makes anything more modern than the Respeaker 4-mic array?). My biggest problem is even with AEC, the voice output is triggering the VAD and so it continually thinks it's getting interrupted by a human. My next attempt will be to try to only signal true VAD if there's also sound coming from anywhere but behind, where the speaker is. It's been an interesting challenge so far though.

Re: Transcribro: On-device Accurate Speech-to-text

#32

Post by refulgentis »

opprobium wrote: post #24
refulgentis wrote: post #17

Earlier quoted context omitted.

Gotcha, then, it's "not even wrong" in the Pauli sense to say Whisper isn't streaming

It is not streaming in the way people normally use this term. It's a fuzzy notion but typically streaming means something encompassing: - Processing and emitting results on something closer to word by word level - Allowing partial results while the user is still speaking and mid-segment - Not relying on an external segmenter to determine the chunking (and therefore also latency) of the output.

This is fascinating because if your hint in another comment indicates you worked on this at Google, it's entirely possible I have this all wrong because I'm missing the actual ML part - I wrote the client encoder & server decoder for Opus and the client-side UI for the SODA launch, and I'm honestly really surprised to hear Google has different stuff. The client-side code loop AGSA used is 100% replicated, in my experience, by using Whisper.

I don't want to make too strong of claims given NDAs (in reality, my failing memory :P) but I'm 99% sure inference on-device is just as fast as SODA. I don't know what to say because I'm flummoxed, it makes sense to me that Whisper isn't as good as SODA, and I don't want to start banging the table about that its no different from a user or client perspective, I don't think that's fair. There's a difference in model architecture and it matters. I think its at least a couple WER behind.

But then where's the better STT solutions? Are all the obviously much better solutions really all locked up? Picovoice is the only closed solution I know of available for local dev, and per even them, it's only better than the worst Whisper. Smallest is 70 MB in ONNX vs. 130 MB for next step up, both inference fine with ~600 ms latency from audio byte to mic to text on screen, ranging from WASM in web browser to 3 year old Android phone.

Re: Transcribro: On-device Accurate Speech-to-text

#33

Post by refulgentis »

Nimitz14 wrote: post #30
refulgentis wrote: post #25

Earlier quoted context omitted.

The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally. Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative c…

This is a complete non sequitur lol. FYI whisper is not a streaming model though it can, with some work, be adapted to be one.

You and I agree fully, then. IMHO it's not too much work, at all, 400 LOC and someone else's models. Of course, as in that old saw, the art is knowing exactly those models, knowing what ONNX is, etc. etc., that's what makes it fast.

The non-sequitor is because I can't feel out what's going on from their perspective, the hedging left a huge range where they could have been saying "I saw the gpt4o demo and theres another way that lets you have more natural conversation" and "hey think like an LSTM model, like Silero, there are voice recognizers that let you magically get a state and current transcription out", or in between, "yeah in reality the models are f(audio bytes) => transcription", which appears to be closer to your position, given your "it's not a streaming model, though it can be adapted"

Re: Transcribro: On-device Accurate Speech-to-text

#34

Post by refulgentis »

r2_pilot wrote: post #31
refulgentis wrote: post #25

Earlier quoted context omitted.

The only models that do what you're poking at hostically are 4o (claimed) and that french company with the 7B one. They're also bleeding edge, either unreleased or released and way wilder, ex. The french one interrupts too much, and screams back in an alien language occasionally. Until these, you'd use echo cancellation to try and allow interruptible dialogue, and thats unsolved, you need a consistently cooperative c…

Thank you for your insight. It confirms some of my suspicions working in this area (you wouldn't happen to know anybody who makes anything more modern than the Respeaker 4-mic array?). My biggest problem is even with AEC, the voice output is triggering the VAD and so it continually thinks it's getting interrupted by a human. My next attempt will be to try to only signal true VAD if there's also sound coming from anyw…

Re: mic, alas, no, BigCo kinda sucked, I had to go way out of my way to get work on interesting stuff, it never mattered, and even when you did, you never got over the immediate wall of your own org, except for brief moments. i.e. never ever had anyone even close to knowing anything about the microphones we'd be using, they were shocked to hear what AEC was, even when what we were working on was a marketing tentpole for Pixel. Funny place.

I'm really glad you saw this. So, so, so much time and hope was wasted there on the Nth team of XX people saying "how hard can it be? given physics and a lil ML, we can do $X", and inevitably reality was far more complicated, and it's important to me to talk about it so other people get a sense it's not them, it's the problem. Even unlimited resources and your Nth fresh try can fail.

FWIW my mind's been grinding on how I'd get my little Silero x Whisper gAssistant on device replica pulling off something akin to the gpt4o demo. I keep coming back to speaker ID: replace Silero with some newer models I'm seeing hit ONNX. Super handwave-y, but I can't help thinking this does an end-around both AEC being shit on presumably most non-Apple devices, and poor interactions from trying to juggle two things operating differently (VAD and AEC). """Just""" detect when there's >= 2 simultaneous speakers with > 20% confidence --- of course, tons of bits missing from there, ideally you'd be resilient to ex. TV in background. Sigh. Tough problems.

Re: Transcribro: On-device Accurate Speech-to-text

#35

Post by Grimblewald »

SparkyMcUnicorn wrote: post #27
yewenjie wrote: post #21

Seems like Gboard is incompatible with it. Is there a good enough open source alternative to Gboard in 2024 that has smooth glide-typing and a similar layout?

Any of these should work. https://github.com/Helium314/HeliBoard https://github.com/openboard-team/openboard https://github.com/rkkr/simple-keyboard (guessing, since AOSP Keyboard works and this is a fork) Not open source: https://www.microsoft.com/en-us/swiftkey Does not have glide/swipe (reserved for symbols), but I just installed and giving it a shot: https://github.com/Julow/Unexpected-Keyboard

Unexpected keyboard is unexpectedly awesome. Looks a bit dated, but boy does it have some functionality packed into it.

Re: Transcribro: On-device Accurate Speech-to-text

#38

Post by cee_el123 »

tmaly wrote: post #36

I wish there was something where I could transcribe iPhone voice memos to text. I would pay for an app that did this.

Google has an app called live transcribe on Android but there's no iPhone version

This is an unaffiliated version looks like https://apps.apple.com/us/app/live-transcribe/id1471473738

Re: Transcribro: On-device Accurate Speech-to-text

#40

Post by mijoharas »

crancher wrote: post #2

Accrescent hype is comically overdone.

I only just saw it from this project.

I see the features listed[0] which seems like a reasonable feature set, but nothing unusual afaict.

If there has been a lot of hype can you tell me what people find compelling about it?

[0] https://accrescent.app/

Post reply on HN