Author here. I ship both Apple speech engines plus WhisperKit side by side in a transcription app, which made it possible to run all five through identical production code on the same audio: LibriSpeech test-clean and test-other, 5,559 utterances, fully on-device on an M2 Pro. Apple published no accuracy numbers for SpeechAnalyzer (or for SFSpeechRecognizer, ever, as far as I can tell), so the migration question has…
Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
21–30 of 251 posts
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#22Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#23Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#24this is amazing. if i had a mac i would try to reverse engineer the code, extract the weights and port it to something that works on linux/windows like torch or burn. then put the code on github and weights on a torrent site. lifes too short to let apple keep their models exclusive.
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#25Whisper small/tiny/base are almost four years old (they were not updated for Whisper v2 or v3). Is there really nothing better to benchmark against by now?
There's tons, Parakeet was the last I remember seeing which seemed to gain traction (independent lightweight implementations etc).
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#26Whisper small/tiny/base are almost four years old (they were not updated for Whisper v2 or v3). Is there really nothing better to benchmark against by now?
There's tons, Parakeet was the last I remember seeing which seemed to gain traction (independent lightweight implementations etc).
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#27Im hoping Apple gets the new Siri working better on older phones. I was excited to use it but the latest beta / Siri runs too slow on my iPhone Pro Max 15. Im looking for the same experience I have when talking to chatGPT. As for past two years or more talking to GPT within it's app and on my iPhone Pro Max 15 it runs smooth as butter :-). This is the experience I was and still am hoping with Apple, but Im thinking a…
Apple would never do that, if anything they did not offer their Siri with the most advanced AI on iPhone 16 Pro Max, which is one year-old only.
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#28Im hoping Apple gets the new Siri working better on older phones. I was excited to use it but the latest beta / Siri runs too slow on my iPhone Pro Max 15. Im looking for the same experience I have when talking to chatGPT. As for past two years or more talking to GPT within it's app and on my iPhone Pro Max 15 it runs smooth as butter :-). This is the experience I was and still am hoping with Apple, but Im thinking a…
Yeah, ChatGPT voice is great experience vs. Siri on that phone. In case you haven't done something like this already: 1. In Shortcuts app, make shortcut named "AI Voice Mode" (or whatever you want, YMMV) 2. Set it to run the ChatGPT action "Voice Mode" (requires at least the minimum paid tier, I think) 3. To trigger, say "Hey Siri, AI Voice Mode" (or whatever you called the shortcut) This is a pretty slick integratio…
Thanks for the tip and if Im not mistaken it's similar to asking Siri to ask chatGPT to ask XYZ?
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#29Still nothing beats OpenAI's VTT. Anthropic's sucks and Apple's isn't even usable. Edit: Getting downvoted by Apple fanboys for telling the truth is a badge of honor.
Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
#30Just ran it against Whisper-Large-V2 on a math lecture (my primary use case for ASR is subtitling math lectures), and it was substantially faster and only slightly worse. Very usable for live transcription though I'll probably stick with whisper for the time being since I don't really need the subtitles to be generated in real time.
If it was faster but worse, maybe compare it to a smaller whisper model?