Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

github.com

Post reply on HN
Print view View original

161–167 of 167 posts

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#161

Post by coder543 »

solarkraft wrote: post #160
coder543 wrote: post #25

Earlier quoted context omitted.

> Siri/iOS-Dictation is truly good when it comes to understanding the speech. What...? It is terrible, even compared to Whisper Tiny , which was released years ago under an Apache 2.0 license so Apple could have adopted it instantly and integrated it into their devices. The bigger Whisper models are far better, and Parakeet TDT V2 (English) / V3 (Multilingual) are quite impressive and very fast. I have no idea what w…

Its quality isn’t great, but it is damn fast and that matters a lot! Whisper doesn’t even work live without hacks.

Parakeet is insanely fast and much more accurate, and it doesn't really matter that Whisper requires hacks to work live when those hacks have existed for years and work great. (The Hello Transcribe app on iOS is a great example of how well Whisper can work with live streaming on an iPhone. The smaller models are extremely fast, even with the "hacks".)

Parakeet TDT's architecture is actually a really cool way to boost both the speed and efficiency of real time STT compared to traditional approaches.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#162

Post by mhamann »

Can you help me understand MetalRT a bit more? Based on the name, it sounds like something that's Apple-only (although, Apple basically co-opted the name Metal, which was traditionally more generic). Does or will MetalRT run on more platforms?

What about MetalRT's relationship to llama.cpp, onnx, MLX, transformers, etc? Is MetalRT a replacement for those? Designed to be compatible with a wide variety of model formats? Or are you just providing an abstraction on top of these?

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#163

Post by alfanick »

swindmill wrote: post #11
alfanick wrote: post #9

I'm not looking for STT->AI ->TTS, I'm looking for truly good voice-to-text experience* on Linux (and others). Siri/iOS-Dictation is truly good when it comes to understanding the speech. Something this level on Linux (and others) would be great, yeah always listening, maybe sending the data somewhere, but give me UX - hidden latency, optimizing for first chars recognized - a good (virtual) input device.

Have you tried https://handy.computer ?

Not bad, almost checks all the marks I want. A) Good quality, locally run model, and surprisingly fast and working on my CPU. B) It transcribes after the session is finished (aka stopped push-to-talk, or after stopping the listening). C) Ha nice, post-processing. D) Still not solved, truly realtime transcription with latency hiding - start typing as soon as you recognize sounds (or after some logical pause, i.e. at the end of sentence). E) Written in Rust, with web-browser config ui. F) Global shortcuts are super finnicky, doesn't recognize my default "Mic" button, fair enough, let me remap to some unused F24... Doesn't recognize F24 due to missing keycode.

It's there, doesn't feel native though. Good integration, not great though (Linux Mint/Cinnamon).

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#164

Post by AmanSwar »

mhamann wrote: post #162

Can you help me understand MetalRT a bit more? Based on the name, it sounds like something that's Apple-only (although, Apple basically co-opted the name Metal, which was traditionally more generic). Does or will MetalRT run on more platforms? What about MetalRT's relationship to llama.cpp, onnx, MLX, transformers, etc? Is MetalRT a replacement for those? Designed to be compatible with a wide variety of model formats…

MetalRT is metal only inference engine (we are making for other hardwares too). Think of it like SGLang or vLLM but for single batch inference on apple silicon. See this blogpost : https://www.runanywhere.ai/blog/metalrt-speech-fastest-stt-t...

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#165

Post by hudtaylor »

shubham2802 wrote: post #76
stingraycharles wrote: post #4

I’m a bit confused by what you’re offering. Is it a voice assistant / AI as described on your GitHub? Or is it more general purpose / LLM ? How does the RAG fit in, a voice-to-RAG seems a bit random as a feature? I don’t mean to come across as dismissive, I’m genuinely confused as to what you’re offering.

RunAnywhere builds software that makes AI models run fast locally on devices instead of sending requests to the cloud. Right now, our focus is Apple Silicon. Today there are two parts: MetalRT - our proprietary inference engine for Apple Silicon. It speeds up local LLM, speech-to-text, and text-to-speech workloads. We’re expanding model coverage over time, with more modalities and broader support coming next. RCLI -…

[dead]

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#167

Post by catlifeonmars »

coder543 wrote: post #124
catlifeonmars wrote: post #123

Earlier quoted context omitted.

> I have a neutral American accent This is tangential but is _any_ accent objectively neutral?

Neutral here means not strongly identifiable as any particular regional American accent. Some people have very strong regional accents, some don’t. It is still clearly an American accent, not British or anything else.

TIL (assuming you mean American as in the United States of America) that this is known as a General American accent.
Post reply on HN