Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speech Recognition and TTS in less than 500kb

github.com

Post reply on HN
Print view View original

41–50 of 101 posts

Re: Speech Recognition and TTS in less than 500kb

#41

Post by nutanc »

This is awesome. I am trying to build a full scale ASR system within 20-25MB. Now that we have Claude code to run experiments, I have started running some experiments. Promising results so far. First realization is that you can capture the nuances of speech in just 3300 embedding vectors(786d). This sequence can be decoded with a small CTC system to get text. Next experiments are on reducing the 768 dimension space into a 64D space. Thats also show some promising results. Hooking up my system so that the agent blogs the results everyday[1]. So my research "claw" setup does the experiments and posts results which I check in the morning and adjust the experiment direction as needed. Its not fully automated yet, but almost there.

[1] https://blog.trulm.com/posts/speech-as-independent-parts/

Re: Speech Recognition and TTS in less than 500kb

#42

Post by pxx »

JSR_FDED wrote: post #40
sgt wrote: post #3

Quick link to the video where he demos it: https://www.youtube.com/watch?v=kMliOFYBiz4

Amazing that this works. As an aside, and I appreciate this is just a demo, if the use case is to get a device to join a WiFi network - would a single or double line lcd with 3 buttons not be cheaper than 520KB?

the target rp2350 is a sub-$1 chip. a 16x2 LCD module is over $1. but more importantly, you might have this much ram sitting around unused on whatever you're building anyway.

Re: Speech Recognition and TTS in less than 500kb

#43

Post by almogo »

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts.

Fishing for anecdotes here, does anyone have any good tts/stt experiences?

Re: Speech Recognition and TTS in less than 500kb

#44

Post by lunixbochs »

nutanc wrote: post #41

This is awesome. I am trying to build a full scale ASR system within 20-25MB. Now that we have Claude code to run experiments, I have started running some experiments. Promising results so far. First realization is that you can capture the nuances of speech in just 3300 embedding vectors(786d). This sequence can be decoded with a small CTC system to get text. Next experiments are on reducing the 768 dimension space i…

I think Google's Conformer paper is SOTA at the I think my small Talon models are next, around 3% lsc clean at ~28M (greedy CTC decoding, no external encoder, no LM, not trained in a domain specific way). I reached around 6.5% at 10M.

I've been working on some new baselines I want to release soon as public artifacts. This article is inspiring me to try pushing the param size down a bit. I suspect we can do large vocabulary end to end in the <5M range.

Re: Speech Recognition and TTS in less than 500kb

#45

Post by andai »

sgt wrote: post #3

Quick link to the video where he demos it: https://www.youtube.com/watch?v=kMliOFYBiz4

Is that Microsoft Sam? :)

(Also, I know it's besides the point but this might be the most painful way to connect to Wifi physically possible. "Make normal everyday tasks slow, tedious and painful" is a bit of an odd choice for a product demo.)

Say, speaking of Sam, what were the memory requirements for SAM (Software Automatic Mouth) on C64. I guess they were not more than 64K? Although, the bulk here is probably for the speech recognition, not the TTS. (And this one does sound a little nicer :)

Browser demo of a reversed SAM:

https://discordier.github.io/sam/index.html

Re: Speech Recognition and TTS in less than 500kb

#47

Post by k9294 »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I'm founder of ottex.ai, I use stt pretty much all the time when work with AI and quite often for communications to draft emails and chat messages.

I started ottex half a year ago after I tested gemini 2.5 flash native audio support. I was blown away by the quality of transcripts and decided to built an app to use it myself.

Currently the default model in the app is Gemini 3 flash, but you can connect to 9 providers and God knows how many models to play with.

I would suggest you to try this models for ai prompting:

- Gemini 3 / 3.5 flash - Soniox rtt v5 - Mistral transcribe v2 - assembly 3.5 pro

Re: Speech Recognition and TTS in less than 500kb

#48

Post by DrSiemer »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

One of my side projects is a tool that lets you control your entire system with STT. It's built on Whisper and supports hot swapping custom profiles, so you can add easy commands for any software.

I intend to use it to work on low stakes vibe coding projects while I'm doing other stuff. Todays LLMs are a lot better at interpreting rambling dictation with mid-message corrections.

There are a few paid programs out there that do the same, but they made my vibe slop sense tingle and are not aimed at development.

Re: Speech Recognition and TTS in less than 500kb

#49

Post by arend321 »

almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

Re: Speech Recognition and TTS in less than 500kb

#50

Post by fragmede »

arend321 wrote: post #49
almogo wrote: post #43

Stt/tts systems always seem to me so promising, but I pretty much never use voice to interface with a computer. Sometimes instead of typing on my phone, I use a voice dictation. I would be keen to use voice to control Claude code, but I've always felt that the way I speak is different from the way I write good prompts. Fishing for anecdotes here, does anyone have any good tts/stt experiences?

I do a ton of coding (codex) with a tts/stt wrapper. During walks, cycling, in the car. Not every task is suited to this style of interaction, but many are. Long form codex replies are condensed, code blocks are suppressed all in the name of making it work for tts feedback. So it works best on well defined projects with guardrails, where you know the agent can perform well.

During cycling! Do you have a phone mount on your bike that you use while biking or is it all in-ear?
Post reply on HN