Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handy – Free open source speech-to-text app

github.com

Post reply on HN
Print view View original

31–40 of 118 posts

Re: Handy – Free open source speech-to-text app

#31

Post by jborichevskiy »

mrroryflint wrote: post #27

On a M4 Macbook Air, there was enough lag to make it unusable for me. I hit the shortcut and start speaking but there was always a 1-2sec delay before it would actually start transcribing even if the icon was displayed.

Curious if you were using AirPods or other Bluetooth headphones for this?

If so, there should be "keep microphone on" or similar setting in the config that may help with this, alternatively, I set my microphone to my MacBook mic so that my headphones aren't involved at all and there is much less latency on activation

Re: Handy – Free open source speech-to-text app

#32

Post by sipjca »

mrroryflint wrote: post #27

On a M4 Macbook Air, there was enough lag to make it unusable for me. I hit the shortcut and start speaking but there was always a 1-2sec delay before it would actually start transcribing even if the icon was displayed.

What microphone are you using?

Re: Handy – Free open source speech-to-text app

#33

Post by blutoot »

I have dystonia which often stiffens my arms in a way that makes it impossible for me to type on a keyboard. TTS apps like SuperWhisper have proven to be very helpful for me in such situations. I am hoping to get a similar experience out of "Handy" (very apt maming from my perspective).

I do, however, wonder if there is a way all these TTS tools can get to the next level. The generated text should not be just a verbatim copy of what I just said, but depending on the context, it should elaborate. For example, if my cursor is actively inside an editor/IDE with some code, my coding-related verbal prompts should actually generate the right/desired code in that IDE.

Perhaps this is a bit of combining TTS with computer-use.

Re: Handy – Free open source speech-to-text app

#34

Post by sipjca »

frankdilo wrote: post #8

This looks great! What’s missing for me to switch from something like Wispr Flow is the ability to provide a dictionary for commonly mistaken words (name of your company, people, code libraries).

There’s a PR for this which will be pulled in soon enough, I can kick off a build of the PR if you want to download a pre release version

Re: Handy – Free open source speech-to-text app

#36

Post by hasperdi »

blutoot wrote: post #33

I have dystonia which often stiffens my arms in a way that makes it impossible for me to type on a keyboard. TTS apps like SuperWhisper have proven to be very helpful for me in such situations. I am hoping to get a similar experience out of "Handy" (very apt maming from my perspective). I do, however, wonder if there is a way all these TTS tools can get to the next level. The generated text should not be just a verba…

What you said is possible by feeding the output of speech-to-text tools into an LLM. You can prompt the LLM to make sense of what you're trying to achieve and create sets of actions. With a CLI it’s trivial, you can have your verbal command translated into working shell commands. With a GUI it’s slightly more complicated because the LLM agent needs to know what you see on the screen, etc.

That CLI bit I mentioned earlier is already possible. For instance, on macOS there’s an app called MacWhisper that can send dictation output to an OpenAI‑compatible endpoint.

Re: Handy – Free open source speech-to-text app

#37

Post by sipjca »

blutoot wrote: post #33

I have dystonia which often stiffens my arms in a way that makes it impossible for me to type on a keyboard. TTS apps like SuperWhisper have proven to be very helpful for me in such situations. I am hoping to get a similar experience out of "Handy" (very apt maming from my perspective). I do, however, wonder if there is a way all these TTS tools can get to the next level. The generated text should not be just a verba…

I totally agree with you and largely what you’re describing is one of the reasons I made Handy open source. I really want to see something like this and see someone go experiment with making it happen. I did hear some people playing with using some small local models (moondream, qwen) to get some more context of the computer itself

I initially had a ton of keyboard shortcuts in handy for myself when I had a broken finger and was in a cast. It let me play with the simplest form of this contextual thing, as shortcuts could effectively be mapped to certain apps with very clear uses cases

Re: Handy – Free open source speech-to-text app

#38

Post by sipjca »

hasperdi wrote: post #36
blutoot wrote: post #33

I have dystonia which often stiffens my arms in a way that makes it impossible for me to type on a keyboard. TTS apps like SuperWhisper have proven to be very helpful for me in such situations. I am hoping to get a similar experience out of "Handy" (very apt maming from my perspective). I do, however, wonder if there is a way all these TTS tools can get to the next level. The generated text should not be just a verba…

What you said is possible by feeding the output of speech-to-text tools into an LLM. You can prompt the LLM to make sense of what you're trying to achieve and create sets of actions. With a CLI it’s trivial, you can have your verbal command translated into working shell commands. With a GUI it’s slightly more complicated because the LLM agent needs to know what you see on the screen, etc. That CLI bit I mentioned ear…

Handy can post process with LLMs too! It’s just currently hidden behind a debug menu as an alpha feature (ctrl/cmd+shift+d)

Re: Handy – Free open source speech-to-text app

#40

Post by bogtap82 »

bn-usd-mistake wrote: post #39

Does anyone have a similar mobile application that works locally and is not too expensive? Mostly looking to transcribe voice messages sent over Signal which does not offer this OOTB

There is one single app I've been able to find that offers Parakeet-v3 for free locally and it's called Spokenly. They have paid cloud models available as well, but the local Parakeet-v3 implementation is totally free and is the best STT has to offer these days regardless. Super fast and accurate. I consider single-user STT basically a solved problem at this point.
Post reply on HN