Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speech Recognition and TTS in less than 500kb

github.com

Post reply on HN
Print view View original

71–80 of 101 posts

Re: Speech Recognition and TTS in less than 500kb

#71

Post by jodrellblank »

schoen wrote: post #38
dwa3592 wrote: post #20

this is good to see. i also trained a stt under 500kb for sub dollar chips. it had about 20 words that it could understand(like start, stop, left, right, go, up etc) and then the spell mode where you could say the word spell and then say the individual english alphabets and close with spell. it was super fun to work on. these tend to be extremely unstable though, like confusion between p and t (at least for my accent…

Could you get people to use the NATO phonetic alphabet for the spelling part? I suppose a challenge is that many people don't know the whole thing, even if they're aware it exists.

NATO phonetic is to be understandable over a noisy radio channel, if you want just distinct sounds then Talon Voice users settled on shorter ones easier to use all the time:

    air a
    bat b
    cap c
    drum d
    each e
    fine f
    gust g
    harp h
    sit i
    jury j
    crunch k
    look l
    made m
    near n
    odd o
    pit p
    quench q
    red r
    sun s
    trap t
    urge u
    vest v
    whale w
    plex x
    yank y
    zip z

Re: Speech Recognition and TTS in less than 500kb

#72

Post by paweladamczuk »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

On some days, half of my promts to Copilot at work are spoken to a local whisper medium model running on an Intel ARC GPU.

Re: Speech Recognition and TTS in less than 500kb

#74

Post by close2 »

jodrellblank wrote: post #71
schoen wrote: post #38

Earlier quoted context omitted.

Could you get people to use the NATO phonetic alphabet for the spelling part? I suppose a challenge is that many people don't know the whole thing, even if they're aware it exists.

NATO phonetic is to be understandable over a noisy radio channel, if you want just distinct sounds then Talon Voice users settled on shorter ones easier to use all the time: air a bat b cap c drum d each e fine f gust g harp h sit i jury j crunch k look l made m near n odd o pit p quench q red r sun s trap t urge u vest v whale w plex x yank y zip z

Interesting, that some words don't start with the letter they represent.

Re: Speech Recognition and TTS in less than 500kb

#75

Post by Avicebron »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I (well CC and I) wrote a tts/stt pipeline for the CLI of CC. It's a lot more, immersive I guess, when I open my dev environment and it gives me a verbal walk through of what's going on.

(this inspired some more demo-y stuff I have where claude can manipulate the mouse and audit things it's built visually in conjunction with that). I'm sure this has already been wrapped up into some MCP framework, but it was fun to build it super early on and it just sort of works for me.

I don't use this in my day job, but it does feel very futuristic when I pull up my home lab.

Re: Speech Recognition and TTS in less than 500kb

#76

Post by sbarre »

kleiba2 wrote: post #63
almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I've always dreamed of having the ability to just talk to my computer (in the right circumstances) so I actually worked in the field for many years. The main reason I never use speech recognition today is because I have zero interest of sending recordings of my voice to the servers of some global corporations. Running speech recognition and TTS locally is quite feasible, as projects like this one show.

If you want a local and open source option (MacOS only at the moment though), I've been happily using Keyscribe for dictation, which is built on Moonshine I believe.

https://rsperko.github.io/keyscribe/

Re: Speech Recognition and TTS in less than 500kb

#77

Post by HarHarVeryFunny »

Joel_Mckay wrote: post #55
HarHarVeryFunny wrote: post #35

Earlier quoted context omitted.

Presumably it's not, but the TTS voice in the video sounds to me more like formant synthesis than diphone - it reminds me of my DECtalk. The project credits does mention espeak (which is formant based) as well as various other TTS projects, although it sounds like they are only using the pronunciation part of espeak, not the voice synthesis. https://github.com/moonshine-ai/moonshine#acknowledgements

It certainly sounds similar, but seems more nimble with phonetic pronunciation in the demo. Having it run on a pico would be pretty impressive =3 http://cmuflite.org/ https://github.com/festvox/flite

> Having it run on a pico would be pretty impressive

Yes, although relative to the DECTalk DTC01, a Pi Pico is a beast !

Pico : dual core ARM @ 133 MHz, 2MB flash, 264K RAM

DECTalk: 68000 @ 10 MHz + TMS 32010 @ 20 MHz (5 MIPS), 256K ROM, 64K RAM

Re: Speech Recognition and TTS in less than 500kb

#78

Post by MiloLeo »

mattmanser wrote: post #62
arend321 wrote: post #49

Earlier quoted context omitted.

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

This is extremely dangerous and you should stop doing it. https://etsc.eu/tiny-proportion-of-drivers-understand-danger...

Talking on the phone is extremely dangerous? What are your thoughts on talking with a passenger? Is that something that people ought to stop doing? Drive in silence. 100% focus?

Seems like an overreaction.

Re: Speech Recognition and TTS in less than 500kb

#79

Post by d4rkp4ttern »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I use the Hex[1] app on MacOs for near instant transcription with Parakeet V3. This is how I speak to Code agents at least 80% of the time. The idea is - I tend to be lazy if I need to type lots of details, so speaking lets me get into details that I otherwise wouldn’t and this helps give more useful context to the agent. This often tends to be an unstructured brain dump so I sometimes ask the agent to repeat back what it understood, so I can make sure, and this also likely helps it stay on track.

[1] hex https://github.com/kitlangton/Hex

I used to use Handy but as of a few months ago it had stuttering and other issues so I switched to Hex. Even for relatively long minute long dictation, transcription is near instant and accuracy is more than sufficient, especially when talking to AI since it that can “read between the lines “.

Using voice with code agents is a huge unlock and I’m surprised to see some people I recommend it to, still resist it.

As for TTS I found it fun to make a voice plugin for Claude Code that uses PocketTTS, so it can give brief updates whenever it finishes a turn:

https://pchalasani.github.io/claude-code-tools/plugins-detai...

Re: Speech Recognition and TTS in less than 500kb

#80

Post by arend321 »

VBprogrammer wrote: post #64
arend321 wrote: post #49

Earlier quoted context omitted.

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

I can't honestly think of a case where this would be remotely useful. This goes somehow beyond vibe coding to vibe interaction, where the only feedback comes via the AI. I'd love to see a concrete example of this working practice.

I've been building out the Android app that works as the wrapper between codex and me this way. ADB debugging access over WireGuard. I ask to add a local wake phrase detection for the phrase "Hey codex" to start dictation. Codex tells me it thinks Vosk or OpenWakeWord might be a good fit, I voice select Vosk. Codex starts working. Codex deploys (updates) the app on my phone and restarts it and I hear the task complete response. I say "Hey Codex" to test and ask for some tuning/improvements, or move onto the next task. All while keeping my phone in my pocket.

But there are countless projects I work on this way. Eg, I got an email from person X, it says it encountered a bug when doing x and y. Codex reads the email (using nvidia/gliner-PII to hide PII data) and investigates the reason for the bug and proposes a solution. I ask it to implement the fix on my dev server and increase test coverage. I enjoy my walk and after say 7 minutes get a an overview of the fix and can decide to deploy to production.

Post reply on HN