Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speech Recognition and TTS in less than 500kb

github.com

Post reply on HN
Print view View original

51–60 of 101 posts

Re: Speech Recognition and TTS in less than 500kb

#51

Post by fragmede »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

For STT, wispr flow has a generous free tier. For TTS, I have Claude read out loud what it just finished as a stop hook, so I know which claude finished up.

Re: Speech Recognition and TTS in less than 500kb

#52

Post by Joel_Mckay »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

Industry leading Interactive Voice Response systems have become very good at filling in ambiguous information from context, and modulating pronunciation to Ape emotional information.

However, being able to interact with these natural language systems in uncontrolled settings is still a fools errand. For STT, there is also regional dialect, slang, and individual differences.

Witnessing blind users hit unrecognizable reading-speeds on old Gordon 8 TTS systems was surprising. I learned people adapt to imperfect systems pretty quickly. =3

Re: Speech Recognition and TTS in less than 500kb

#53

Post by cafebabbe »

fragmede wrote: post #50
arend321 wrote: post #49

Earlier quoted context omitted.

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

During cycling! Do you have a phone mount on your bike that you use while biking or is it all in-ear?

... I cannot think of an activity less suitable for coding (except scuba diving)

I would die In minutes

Re: Speech Recognition and TTS in less than 500kb

#55

Post by Joel_Mckay »

HarHarVeryFunny wrote: post #35
userbinator wrote: post #24

This looks like an extreme point for AI-based TTS, as formant/tract modeling synths tend to be more accurate if you want TTS in a tiny amount of compute, but sound distinctly robotic. TTS (neural diphone synth @ 16 kHz) ~1.8 MiB voice pack This is in the realm of Microsoft Sam.

Presumably it's not, but the TTS voice in the video sounds to me more like formant synthesis than diphone - it reminds me of my DECtalk. The project credits does mention espeak (which is formant based) as well as various other TTS projects, although it sounds like they are only using the pronunciation part of espeak, not the voice synthesis. https://github.com/moonshine-ai/moonshine#acknowledgements

It certainly sounds similar, but seems more nimble with phonetic pronunciation in the demo.

Having it run on a pico would be pretty impressive =3

http://cmuflite.org/

https://github.com/festvox/flite

Re: Speech Recognition and TTS in less than 500kb

#56

Post by raphlinus »

andai wrote: post #45
sgt wrote: post #3

Quick link to the video where he demos it: https://www.youtube.com/watch?v=kMliOFYBiz4

Is that Microsoft Sam? :) (Also, I know it's besides the point but this might be the most painful way to connect to Wifi physically possible. "Make normal everyday tasks slow, tedious and painful" is a bit of an odd choice for a product demo.) Say, speaking of Sam, what were the memory requirements for SAM (Software Automatic Mouth) on C64. I guess they were not more than 64K? Although, the bulk here is probably for…

There are two Sams here, the Microsoft one and the C64 one. I don't believe there's any connection between the two other than the name.

According to [1], the weight of a modern runnable version is around 39k.

The ratio of how good it sounds compared to how much computing power it uses is ridiculous. The C64 has ballpark 3 orders of magnitude less CPU throughput as an RP2350, and the codebase uses an impressive array of tricks to do actual formant synthesis (barely) and a pretty refined form of Elovitz text to phoneme conversion. One of my favorite tricks is its up and down bouncy pitch, which is not random, but based on the opposite contour as the first formant. It's simplistic but enough to make it not sound like a robotic monotone.

I've been playing around with this some myself and SAM is an inspiration, along with other landmark systems like MITalk (predecessor to DECtalk), SP0256, and other. I believe it's possible to use modern techniques to get pretty good sounding speech in, say, 64k and 10% of the throughput of a RP2350. It's really cool to see projects like OP, especially under permissive license.

[1]: https://simulationcorner.net/index.php?page=sam

Re: Speech Recognition and TTS in less than 500kb

#57

Post by arend321 »

fragmede wrote: post #50
arend321 wrote: post #49

Earlier quoted context omitted.

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

During cycling! Do you have a phone mount on your bike that you use while biking or is it all in-ear?

I have my phone in my pocket, no screen interaction is required. I use a headset (Shokz OpenComm2) with wind muff (when cycling). I made an Android app that listens for codex turn-complete or intermediate updates and plays them back to me. My answer is transcribed and pasted back to the relevant codex (tmux) session on the server (which I can select by voice) a tiny layer helps with things like /new, /plan, answer selection, etc.

Re: Speech Recognition and TTS in less than 500kb

#58

Post by stanko »

This is really impressive.

If I get time, I would like to try compiling it to WASM. This would allow me to swap my robot poet’s native browser voice synthesis for it. Not sure if it is worth it, but it will be fun to play around with.

Edit: typo

[0] https://muffinman.io/bard/

Post reply on HN