Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Voxtral Transcribe 2

mistral.ai

Post reply on HN
Print view View original

241–250 of 255 posts

Re: Voxtral Transcribe 2

#241

Post by tekacs »

drakenot wrote: post #185
tekacs wrote: post #32

Earlier quoted context omitted.

Having built with and tried every voice model over the last three years, real time and non-real time... this is off the charts compared to anything I've seen before. And open weight too! So grateful for this.

This past month Parakeet v3 dropped with a streaming ASR model that is 0.6B params, can run on a CPU and is super good.

I did say all the model. :)

Yes I've tried Parakeet v3 too. For its own purpose - running locally - it's amazing.

The thing that's particularly amazing about this Voxtral model is how incredibly rock solid the accuracy is.

For the longest time previous models have been 'mostly correct' or as people have commented elsewhere on this HN thread, have dropped sentences or lost or added utterances.

I have no affiliation with these folks, but I tried and struggled to get this model to break even speaking as adversariately as I could.

That's a totally different class of model.

Re: Voxtral Transcribe 2

#242

Post by boringg »

ffsm8 wrote: post #70
boringg wrote: post #39

Pseudo related -- am I the only one uncomfortable using my voice with AI for the concern that once it is in the training model it is forever reproducible? As a non-public person it seems like a risk vector (albeit small),

It's a real issue, but why do you only see it in ai? It's true for any case where you're speaking into a microphone Depending on the permissions granted to apps on your mobile device, it can even be passively exfiltrated without you ever noticing - and that's ignoring the video clips people take and put online. Like your grandma uploading to Facebook a short moment from a Christmas meet or similar There have already…

I completely agree - but I think those scams you refer to are less explicit but could potentially be anywhere.

AI I am intentionally providing them my voice. I'm not sure that the value to security risk is good.

Re: Voxtral Transcribe 2

#243

Post by Obertr »

nostrebored wrote: post #206
Obertr wrote: post #204

Very nice! The thing I am missing is turn detection. In real time audio we need the turn detection to understand when AI should speak. Unfortunately this makes it not a complete deepgram replacement yet!

Is deepgram really performing better than open source turn detection models for you? In our tests it is not.

what is SOTA?

Re: Voxtral Transcribe 2

#245

Post by espadrine »

simonw wrote: post #20

This demo is really impressive: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim... Don't be confused if it says "no microphone", the moment you click the record button it will request browser permission and then start working. I spoke fast and dropped in some jargon and it got it all right - I said this and it transcribed it exactly right, WebAssembly spelling included: > Can you tell me about RSS and At…

It is quite impressive.

I have seen the same impressive performance about 7 months ago here: https://kyutai.org/stt

If I look at the architecture of Voxtral 2, it seems to take a page from Kyutai’s delayed stream modeling.

The reason the delay is configurable is that you can delay the stream by a variable number of audio tokens. Each audio token is 80 ms of audio, converted to a spectrogram, fed to a convnet, passed through a transformer audio encoder, and the encoded audio embedding is passed, with a history of 1 audio embedding per 80 ms, into a text transformer, which outputs text embedding, then converted to a text token (which is thus also worth 80ms, but there is a special [STREAMING_PAD] token to skip producing a word).

There is no cross-attention in either Kyutai's STT nor in Voxtral 2, unlike Whisper's encoder-decoder design!

Re: Voxtral Transcribe 2

#246

Post by janalsncm »

idiotsecant wrote: post #136
janalsncm wrote: post #53

I noticed that this model is multilingual and understands 14 languages. For many use cases, we probably only need a single language, and the extra 13 are simply adding extra latency. I believe there will be a trend in the coming years of trimming the fat off of these jack of all trades models. https://aclanthology.org/2025.findings-acl.87/

The hilarious part of this comment is all the comments around it complaining about not supporting enough languages

It’s a little bit like asking for everything to be included in the Standard Library. Sure, it sounds nice at first, but now you need to maintain tons of dependencies. And any time you want to do one thing, you bring along the baggage of every other thing.

Languages are similar. They also change over time. So now if you want to release a v2 you need an updated corpus for all languages. Or if you get access to an updated corpus for a small language, it might not merit a new model version since it’s only one out of the 14.

Re: Voxtral Transcribe 2

#247

Post by fph »

hephaes7us wrote: post #192
antirez wrote: post #148

Earlier quoted context omitted.

I wish I had a Google Keyboard that could easily run on Whisper Medium. This is already great. But unfortunately would be too much inference cost, incredibly slow. The problem with Whisper is not the inference quality: medium and large are incredible. Is that the base model is not enough, and the only one with fast inference in mobile devices.

FUTO keyboard is trying to do this. I think they have some kind of distillation of Whisper running on-device.

They are just shipping the same whisper-small that everyone else is using, and did not much to improve their models since release. Other models have been "coming soon" forever. https://keyboard.futo.org/voice-input-models

Re: Voxtral Transcribe 2

#248

Post by janalsncm »

ryan_lane wrote: post #200
janalsncm wrote: post #53

I noticed that this model is multilingual and understands 14 languages. For many use cases, we probably only need a single language, and the extra 13 are simply adding extra latency. I believe there will be a trend in the coming years of trimming the fat off of these jack of all trades models. https://aclanthology.org/2025.findings-acl.87/

"I only speak one language, so models I use should only understand one".

Engineering is about tradeoffs. If the model is being used in an English-only context then tacking on 13 other languages might not be worth the cost.

You are also implicitly choosing worse performance in English by adding extra languages. So you could have a better monolingual model for the same number of weights.

Re: Voxtral Transcribe 2

#249

Post by jpeeler »

owenbrown wrote: post #216

The other demos didn't work for me, so I made https://github.com/owenbrown/transcribe It's just a python script to test the streaming. Wow, Voxtral is amazing. It will be great when someone stitches this up so an LLM starts thinking, researching for you, before you actually finish talking. Like, create a conversation partner with sub 0.5 second latency. For example, you ask it a multi part questions and, as soon as y…

Yes, appreciate this.

I noticed that with both models voxtral-mini-transcribe-realtime-2602 and voxtral-mini-2602 filler words are ignored. I'd like to be able to count words/sounds, specifically "um" or "uh" for improvement purposes. Any good models that handle that?

Re: Voxtral Transcribe 2

#250

Post by 85392_school »

mdrzn wrote: post #9
dmix wrote: post #7

> At approximately 4% word error rate on FLEURS and $0.003/min Amazons transcription service is $0.024 per minute, pretty big difference https://aws.amazon.com/transcribe/pricing/

Is it 0.003 per minute of audio uploaded, or "compute minute"? For example fal.ai has a Whisper API endpoint priced at "$0.00125 per compute second" which (at 10-25x realtime) is EXTREMELY cheaper than all the competitors.

It can actually go much lower. Gemini costs around $0.01/hour of transcription last time I checked.
Post reply on HN