Live data from Hacker News

Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

get-inscribe.com

161–170 of 251 posts

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#161

Earlier quoted context omitted.

There's tons, Parakeet was the last I remember seeing which seemed to gain traction (independent lightweight implementations etc).

I have tried everything (that will run on a 12GB RTX 4070) and I have yet to find anything with better accuracy than Whisper V2 Large for my dataset (discord audio from TTRPG sessions, isolated per-speaker, mostly non-American accents)

Voxtral to me what better

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#162

Earlier quoted context omitted.

It also struggles with my NYC-area accent, which is only medium thick.

The Two Yoots problem. Do you use d's in place of t's such as dees/dems/dose/dere? I have a heavy queens accent so you'll hear me say things like "deres tree uh dem ova dere."

Surprised to read this as a Queens thing, this sounds perfectly at home in the Midwest.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#163

Earlier quoted context omitted.

Reminds me of the time my neighbours must have wondered if I was having some kind of a breakdown when trying out really basic MacOS voice recognition back in the early 2000s. There was a keyboard shortcut and you could say something like "phone number for firstname lastname" and it would theoretically show you that phone number. Thing is it didn't seem to like a British accent, so I spent a good hour trying out diffe…

Scottish Elevator - Voice Recognition - https://www.youtube.com/watch?v=NMS2VnDveP8

Thanks for this gem, had a good laugh

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#164

Earlier quoted context omitted.

I’m not even using it for diarisation just transcription and it’s amazing. It also doesn’t need a VAD A new VAD I found though is FireRedVAD which has better benchmark results than TEN and Silero by far

Is MOSS a streaming model or only for offline? For that VAD how are you integrating it into a model like Whisper etc?

I'm using it offline. But it's much faster than realtime so it should be usable for streaming. I just asked Codex / Sol to integrate FireRedVAD with Whisper...

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#165
I make an iOS app that uses this API heavily for transcribing diverse audio of varying bitrate and recording quality. The audio often contains music, multiple speakers, sound effects. SpeechAnalyzer almost always gets it.

It can struggle with proper nouns but will return something phonetically similar.

My main gripe is that it requires a separate model download per language. I understand the why they did this (to save disk space). But it makes multi-lingual audio hard to transcribe unless you know ahead of time the languages in the audio.

As an app developer the biggest win from using Apple's model is I don't have to bundle it in my app so my app looks much smaller. If a user has many transcription apps each one could have their own model. If Apple's model is used only one copy is needed.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#166
I took a swing at bringing this into Handy.computer if anybody's interested: https://github.com/cjpais/Handy/discussions/1031 . Looks like there has been past demand for someone to implement it, but no proposed PRs. This article was inspiring.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#167

I use Spokenly, offline-only mode with the Nvidia model. All local, totally free. Highly recommend

For just desktop, I use Handy (open source): https://handy.computer/

And yea, Nvidia's Parakeet v3 is good enough for my own just local transcription most of the time.

When I need local transcription to be more reliable and I don't have the energy to proof read a long ramble, I still often just pop open chatGPT, dictate, cut, paste.

But we're pretty much already to the point where local transcription models can replace cloud ones for personal use. They're still a bit rough around the edges in terms of polish and latency, but plenty of people are fine with that to avoid yet another app subscription and not having to worry about wondering what's potentially happening with their data.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#169
post #155

Earlier quoted context omitted.

You are being naive. An Apple device makes dozens of network requests every minute or so to Apple. It is neigh on impossible to verify what is being requested or sent. Also unplugging the Internet and verifying that something still works does not mean the app won't phone home behind your back when it can. These things are designed to fail silently.

Download use Little Snitch or https://alternativeto.net/software/little-snitch/ https://www.g2.com/products/little-snitch/competitors/altern... There are many alternatives for trying to find out what’s going on. If you don’t want to bother, and most people don’t, well, what else is there to say? It is generally a good idea to know what software is phoning home, if you can pinpoint it. If you have any software recomme…

It's more complex than that. LittleSnitch or other application firewalls won't help when programs like package managers legitimately need to access the Internet. And figuring what what is being sent would require decrypting the traffic. There is no easy way to figure out if Apple or some rogue app is siphoning off your home directory.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#170
post #94
post #64

Earlier quoted context omitted.

Also, this test is English-only, while a strong point of other models is to understand different languages without first having to say which one (so you don't need 3 different keyboard shortcuts if you wanna dictate in 3 languages day-to-day)

Does anyone have any experience with Mandarin STT? What's a good model for this? The use-case I have is subtitling of Mandarin speech.

I use Systran/faster-whisper-medium for real-time subtitling, but you need to get used to the context it's used it and the weirdness it translates into. Parakeet has great mandarin>CN text, but running that + a translation model has been tricky and I never got it fast.
Post reply on HN