Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transcribe.cpp

workshop.cjpais.com

Post reply on HN
Print view View original

71–80 of 180 posts

Re: Transcribe.cpp

#71

Post by jerieljan »

Nice. I did transcriptions on a casual project before that went through something like this. Transcribing videos or audio files with Whisper? Very common. But having to swap it out with Qwen3 or a different family of ASR models? Oops, not as straightforward. For Qwen for example you gotta deal with the forced aligner or it won't be good as subtitles, and then gotta deal with some requirements and considerations if you want to make use of MLX on a Mac or something.

Will definitely check this out since it sounds like it eases through the pain of dealing with these.

Re: Transcribe.cpp

#72

Post by markisus »

The post makes it seem like ONNX is CPU only. I've used ONNX runtime to run models on Nvidia GPUs. The runtime can even dispatch to TensorRT. I'm not sure what the performance is on Apple hardware so maybe that was the motivation for moving away from ONNX.

Re: Transcribe.cpp

#74

Post by sipjca »

markisus wrote: post #72

The post makes it seem like ONNX is CPU only. I've used ONNX runtime to run models on Nvidia GPUs. The runtime can even dispatch to TensorRT. I'm not sure what the performance is on Apple hardware so maybe that was the motivation for moving away from ONNX.

TensorRT and CUDA is effectively the same speed as CPU for the speech to text models I was testing via ONNX at a huge binary bloat penalty. WGPU is hard to ship and also equivalent speed or slower. This may not be the case for LLM or other models but the runtimes did not seem well supported for what I needed to do. ONNX is incredibly well optimized for CPU, best in class even, but the other execution providers at least for STT seemed lacking.

I did this investigation before creating transcribe.cpp it would have been much more convenient and save me literal months of work. Happy to share the repo and binaries produced as well, but it was mostly throw away work to profile how to ship accelerated ONNX in Handy.

Re: Transcribe.cpp

#75

Post by nohup2 »

sneak wrote: post #66
ghm2199 wrote: post #11

Congrats on shipping this. I love handy on my Mac, my phone for STT in situations where it’s not possible/poor performance of the native Model for STT(e.g apple’s thing is not upto scruff, like mistranslating words corresponding to a domain). Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is am…

OS-native dictation on iOS requires uploading your address book to Apple on every request, even if you don’t use iCloud. I unfortunately have to leave it disabled for this reason.

Are you sure? Just tested it and it works locally and offline

Re: Transcribe.cpp

#76

Post by rubidium »

primaprashant wrote: post #64

Handy is an amazing cross-platform app for dictation from the author. There are other awesome open-source dictation tools as well like native macOS ones. You do not need SaaS subscription in this day and age for transcription. I maintain this list of all the best open-source ones in this awesome-style GitHub repo. People looking for open-source dictation tools, hope you find something that works for you here: https:/…

Any that also support translation? How much harder/ easier of a problem is local translation compared to transcription?

Re: Transcribe.cpp

#77

Post by primaprashant »

rubidium wrote: post #76
primaprashant wrote: post #64

Handy is an amazing cross-platform app for dictation from the author. There are other awesome open-source dictation tools as well like native macOS ones. You do not need SaaS subscription in this day and age for transcription. I maintain this list of all the best open-source ones in this awesome-style GitHub repo. People looking for open-source dictation tools, hope you find something that works for you here: https:/…

Any that also support translation? How much harder/ easier of a problem is local translation compared to transcription?

If you're talking about translated text, then that should be super easy. Most of these dictation tool support post-processing with LLM to remove filler words, fix punctuation, etc. I'd imagine you can change the system prompt for the post-processing step to do the translation instead, and you'd get translated text.

Re: Transcribe.cpp

#78

Post by ctas »

I'm using Handy on macOS and love it. Unfortunately, hotkeys still doesn't seem to work on Wayland, which make it unusable.

Re: Transcribe.cpp

#79

Post by nohup2 »

shade wrote: post #12

Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers…

Have you published your app? I would love to take a look and test it

Re: Transcribe.cpp

#80

Post by dostick »

sipjca wrote: post #63
dostick wrote: post #56

Does this support filtering of “umm”,”err”, “ugh”, or that is nit yet possible with open source models?

Not in the library itself, it’s pure inference. Some models have this trained out of them anyhow. Otherwise this is a post processing task which is not really inference

So it looks like something that app would be doing, or run another model over the output to smoothen out and remove these things?
Post reply on HN