Live data from Hacker News

Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

github.com

51–60 of 167 posts

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#51
FWIW this RCLI is only MIT license but their engine MetalRT is commercial. Not sure the license of their models I guess also not MIT. So IMHO this repo is misleading.

Not sure why they decided to reinvent the wheel and write yet another ML engine (MetalRT) which is proprietary. I would most likely bet on CoreML since it have support for ANE (apple NPU) or MLX.

Other popular repos for such tasks I would recommend:

https://github.com/FluidInference/FluidAudio

https://github.com/DePasqualeOrg/mlx-swift-audio

https://github.com/Blaizzy/mlx-audio

https://github.com/k2-fsa/sherpa-onnx

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#52
post #51

FWIW this RCLI is only MIT license but their engine MetalRT is commercial. Not sure the license of their models I guess also not MIT. So IMHO this repo is misleading. Not sure why they decided to reinvent the wheel and write yet another ML engine (MetalRT) which is proprietary. I would most likely bet on CoreML since it have support for ANE (apple NPU) or MLX. Other popular repos for such tasks I would recommend: htt…

Nice list.

What about for on-device RAG use cases?

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#53
post #51

FWIW this RCLI is only MIT license but their engine MetalRT is commercial. Not sure the license of their models I guess also not MIT. So IMHO this repo is misleading. Not sure why they decided to reinvent the wheel and write yet another ML engine (MetalRT) which is proprietary. I would most likely bet on CoreML since it have support for ANE (apple NPU) or MLX. Other popular repos for such tasks I would recommend: htt…

[dead]

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#54
post #51

FWIW this RCLI is only MIT license but their engine MetalRT is commercial. Not sure the license of their models I guess also not MIT. So IMHO this repo is misleading. Not sure why they decided to reinvent the wheel and write yet another ML engine (MetalRT) which is proprietary. I would most likely bet on CoreML since it have support for ANE (apple NPU) or MLX. Other popular repos for such tasks I would recommend: htt…

Updating the readme asap - but thanks for the feedback. Also, please checkout few things : https://www.runanywhere.ai/blog/metalrt-speech-fastest-stt-t... https://www.runanywhere.ai/blog/metalrt-fastest-llm-decode-e...

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#55

This doesn't work on any of the methods I've tried.

Please open the issue - if it's not working ? I believe you should be able to install it via : curl -fsSL https://raw.githubusercontent.com/RunanywhereAI/RCLI/main/in... | bash

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#56
post #9

I'm not looking for STT->AI ->TTS, I'm looking for truly good voice-to-text experience* on Linux (and others). Siri/iOS-Dictation is truly good when it comes to understanding the speech. Something this level on Linux (and others) would be great, yeah always listening, maybe sending the data somewhere, but give me UX - hidden latency, optimizing for first chars recognized - a good (virtual) input device.

> I'm not looking for STT->AI->TTS, I'm looking for truly good voice-to-text experience

Umm, ah, wait no, uhh yes you are. Unless, hang on, you are possessed with greater umm speech capabilities than most, wait nevermind start over. Unless you never make a mistake while talking, you want AI to take out the "three, wait no four" and just leave the output with "four" from what you actually spoke. Depending on your use case.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#58
post #9

I'm not looking for STT->AI ->TTS, I'm looking for truly good voice-to-text experience* on Linux (and others). Siri/iOS-Dictation is truly good when it comes to understanding the speech. Something this level on Linux (and others) would be great, yeah always listening, maybe sending the data somewhere, but give me UX - hidden latency, optimizing for first chars recognized - a good (virtual) input device.

> Siri/iOS-Dictation is truly good when it comes to understanding the speech. What...? It is terrible, even compared to Whisper Tiny , which was released years ago under an Apache 2.0 license so Apple could have adopted it instantly and integrated it into their devices. The bigger Whisper models are far better, and Parakeet TDT V2 (English) / V3 (Multilingual) are quite impressive and very fast. I have no idea what w…

Terrible? It's fine. What's your accent that it's terrible? It even pulls last names from my address book and spells them right.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#59
post #50

Based on the demo video, the TTS sounds like it's 10 years out of date. I would not enjoy interacting with it.

Its kokoro TTS not ours, we have range of options.

Just need some few days to have our catalog of models out soon!!

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#60

Earlier quoted context omitted.

> Siri/iOS-Dictation is truly good when it comes to understanding the speech. What...? It is terrible, even compared to Whisper Tiny , which was released years ago under an Apache 2.0 license so Apple could have adopted it instantly and integrated it into their devices. The bigger Whisper models are far better, and Parakeet TDT V2 (English) / V3 (Multilingual) are quite impressive and very fast. I have no idea what w…

Terrible? It's fine. What's your accent that it's terrible? It even pulls last names from my address book and spells them right.

Terrible relative to everything else that exists today. I have a neutral American accent.

Maybe you just don’t know what you’re missing? Google’s default speech to text is still bad compared to Whisper and Parakeet, but even Google’s is markedly better than Apple’s.

I cannot think of a single speech to text system that I’ve run into in the past 5 years that is less accurate than the one Apple ships.

Sure, Apple’s speech to text is incredible compared to what was on the flip phone I had 20 years ago. Terrible is relative. Much better options exist today, and they’re under very permissive licenses. Apple’s refusal to offer a better, more accessible experience to their users is frustrating when they wouldn’t even have to pay a licensing fee to ship something better. Whisper was released under a permissive license nearly 4 years ago.

Apple also restricts third party keyboards to an absurdly tiny amount of memory, so it isn’t even possible to ship a third party keyboard that provides more accurate on-device speech to text without janky workarounds (requiring the user to open the keyboard's own app first each time).

Post reply on HN