Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

get-inscribe.com

Post reply on HN
Print view View original

31–40 of 251 posts

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#31

Post by paul7986 »

kridsdale1 wrote: post #11
paul7986 wrote: post #7

Im hoping Apple gets the new Siri working better on older phones. I was excited to use it but the latest beta / Siri runs too slow on my iPhone Pro Max 15. Im looking for the same experience I have when talking to chatGPT. As for past two years or more talking to GPT within it's app and on my iPhone Pro Max 15 it runs smooth as butter :-). This is the experience I was and still am hoping with Apple, but Im thinking a…

I’m on iPhone 17 Pro Max, 27 beta 3. New Siri is impressive in that it answers satisfactorily now 80% of the time vs 10% with old Siri. But it’s slow as shit. GPT, Claude, and Gemini can answer me in 5-10 seconds. Google AI Mode can answer in 2 seconds. New Siri usually takes 25 seconds to respond to me. This morning it timed out (with strong network connection) when asked a simple multiplication question.

Damn slow on your newer phone too.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#32

Post by johncoatesdev »

altmanaltman wrote: post #24
tancop wrote: post #8

this is amazing. if i had a mac i would try to reverse engineer the code, extract the weights and port it to something that works on linux/windows like torch or burn. then put the code on github and weights on a torrent site. lifes too short to let apple keep their models exclusive.

Aside from the legality of it, I think you are underestimating how complex it can be to do that. It is possible in theory but not something that will be a fun side quest like you are making it seem.

With a IDA Pro decompiler license & MCP server, paired with Codex/Claude Code... it would be a fun side quest.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#33

Post by seviu »

scuppernong wrote: post #14

Just ran it against Whisper-Large-V2 on a math lecture (my primary use case for ASR is subtitling math lectures), and it was substantially faster and only slightly worse. Very usable for live transcription though I'll probably stick with whisper for the time being since I don't really need the subtitles to be generated in real time.

Been using it for a podcast app I have been developing for half a year lol (I hope I publish it by version 27) and I can confirm it’s real fast.

Splitting the audio in multiple segments and firing it up without hitting the maximum limit of concurrent decoding streams makes it blazing fast. Fair enough you loose the cut, but it’s good enough for just podcast. In one minute it chews through one hour of audio. This on an iPhone 17 Pro.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#34

Post by satvikpendem »

Whisper is the wrong model to benchmark against, or rather, there are better models that are state of the art now like Nemotron and Parakeet both by Nvidia, as well as Mistral's Voxtral and Cohere Transcribe.

However, what's funny is, RIP to a lot of the paid apps that simply wrap Whisper, I'm sure Apple will make a native GUI such as a recorder app for macOS that obviates the need for these wrappers, which everyone seems to be vibe coding these days.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#35

Post by Tsarp »

Any chance you can benchmark against whisper large and large v3 turbo? These run comfortably on older Macbooks and are still far more accurate in real life dictation compared to even the parakeet models( despite ASR leaderboards) with an RTF < 1.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#36

Post by satvikpendem »

seviu wrote: post #33
scuppernong wrote: post #14

Just ran it against Whisper-Large-V2 on a math lecture (my primary use case for ASR is subtitling math lectures), and it was substantially faster and only slightly worse. Very usable for live transcription though I'll probably stick with whisper for the time being since I don't really need the subtitles to be generated in real time.

Been using it for a podcast app I have been developing for half a year lol (I hope I publish it by version 27) and I can confirm it’s real fast. Splitting the audio in multiple segments and firing it up without hitting the maximum limit of concurrent decoding streams makes it blazing fast. Fair enough you loose the cut, but it’s good enough for just podcast. In one minute it chews through one hour of audio. This on a…

What's different about your podcast app?

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#37

Post by satvikpendem »

drnick1 wrote: post #23

If this isn't open source/weights and can't run locally, I don't see how this is a replacement for Whisper or other open models, e.g. within Home Assistant.

It's a local model so it's essentially open weight such that you could feasibly export it somehow since it's already on the laptop somewhere. Apfel is a wrapper app like ChatGPT but using Apple Foundation Models, I assume something similar will happen with this transcription model.

https://apfel.franzai.com/

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#38

Post by satvikpendem »

modeless wrote: post #12

Whisper small/tiny/base are almost four years old (they were not updated for Whisper v2 or v3). Is there really nothing better to benchmark against by now?

There are many [0], you can search and filter by streaming and open weight only as well.

Looks like Voxtral and Nvidia's Nemotron are best.

[0] https://artificialanalysis.ai/speech-to-text/non-streaming

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#39

Post by seviu »

satvikpendem wrote: post #36
seviu wrote: post #33

Earlier quoted context omitted.

Been using it for a podcast app I have been developing for half a year lol (I hope I publish it by version 27) and I can confirm it’s real fast. Splitting the audio in multiple segments and firing it up without hitting the maximum limit of concurrent decoding streams makes it blazing fast. Fair enough you loose the cut, but it’s good enough for just podcast. In one minute it chews through one hour of audio. This on a…

What's different about your podcast app?

Nothing really, except that I get to play with SpeechAnalyzer APIs, foundation models, translations. It’s basically my playground where to try all things. Been listening a lot of Chinese podcasts lately, transcribed and translated by local models.

Edit: all that said, the app is irrelevant. What I want to say is that live transcripts on iOS using Apples frameworks works very well. Only thing I miss is diarization support.

Re: Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

#40

Post by edude03 »

drnick1 wrote: post #23

If this isn't open source/weights and can't run locally, I don't see how this is a replacement for Whisper or other open models, e.g. within Home Assistant.

It's not open weight, but the point is to be an on device (and thus local, privacy preserving) option. The article mentions that as the caveat

> What this means if you just want good transcription

> If you are on a current iPhone or Mac, the best on-device transcription engine for English is already in the operating system, and the private option is no longer the compromise option

Post reply on HN