Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

github.com

Post reply on HN
Print view View original

131–140 of 167 posts

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#131

Post by gigatexal »

shubham2802 wrote: post #127
neya wrote: post #116

Earlier quoted context omitted.

Sorry, but, this is not really a confidence inspiring response. Accepting the mistake and fixing the leak altogether would have been the better way to handle this. This is a developer forum, we all make mistakes. Framing it as bait just sounds like bad PR management. How can we trust your product if you can't fulfil basic security 101? Not being harsh but this kind of lax response for a serious mistake is not accepta…

I see, sure will fix it asap. Again, thanks for feedback.

Yeah wow. These responses to constructive feedback show an immature team full of hubris. This whole thing is DOA to me. Thank you HN for showing me this.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#132

Post by word_saladist »

shubham2802 wrote: post #129
neya wrote: post #116

Earlier quoted context omitted.

Sorry, but, this is not really a confidence inspiring response. Accepting the mistake and fixing the leak altogether would have been the better way to handle this. This is a developer forum, we all make mistakes. Framing it as bait just sounds like bad PR management. How can we trust your product if you can't fulfil basic security 101? Not being harsh but this kind of lax response for a serious mistake is not accepta…

my earlier reply was too glib. Even though the key had no usable balance, it still should not have been exposed. We’re removing it now and fixing the demo flow so this doesn’t happen again. Thanks for calling it out. Cheers!

This is pretty far off from being an intelligible sentence. I wonder if it’s a symptom of people getting used to LLMs being able to parse intent and meaning from fragmentary, disjointed text such as this.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#133

Post by JSR_FDED »

brian-armstrong wrote: post #115

What kind of self-disrespecting dev is using MacOS in TYOOL 2026?

The ones who like using local LLMs

The ones who like top-notch hardware

The ones who build stuff and don’t make a religious issue out of everything

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#134

Post by dajonker »

alfanick wrote: post #9

I'm not looking for STT->AI ->TTS, I'm looking for truly good voice-to-text experience* on Linux (and others). Siri/iOS-Dictation is truly good when it comes to understanding the speech. Something this level on Linux (and others) would be great, yeah always listening, maybe sending the data somewhere, but give me UX - hidden latency, optimizing for first chars recognized - a good (virtual) input device.

I use voxtype on my Linux machine with parakeet. Super fast and regularly even gets the tech lingo correct. You can configure prompts and keywords to help with that as well.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#135

Post by neya »

shubham2802 wrote: post #129
neya wrote: post #116

Earlier quoted context omitted.

Sorry, but, this is not really a confidence inspiring response. Accepting the mistake and fixing the leak altogether would have been the better way to handle this. This is a developer forum, we all make mistakes. Framing it as bait just sounds like bad PR management. How can we trust your product if you can't fulfil basic security 101? Not being harsh but this kind of lax response for a serious mistake is not accepta…

my earlier reply was too glib. Even though the key had no usable balance, it still should not have been exposed. We’re removing it now and fixing the demo flow so this doesn’t happen again. Thanks for calling it out. Cheers!

No worries, like I said, we all make mistakes. Live and learn. All the best.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#136

Post by tomhow »

Imustaskforhelp wrote: post #86
tomhow wrote: post #82

Earlier quoted context omitted.

This post is about the launch of a YC company and its product. It's reasonable that when a company launches a product, the discussion is focused on the product that it is launching. We moderate that way whether it's a YC company's Launch HN or anyone else posting a Show HN. Keeping discussion on-topic is one of the most important things we moderators do, and is the main reason HN is a place where people like to parti…

I think I agree with ya, Although I remember some VPN company related to liberachat ownership where something offtopic wasn't handled such way. In general, I think that if both you and dang say something consistent. Then I do trust you and hackernews moderation for being fairly transparent about it and I really appreciate it. It could very well be that I remember that instant where off-topic=on-topic and considering…

Thanks for sharing your thoughts in such detail. It's an understandable position.

Our position is that we give many people second (and more) chances here, whether or not they're YC founders, startup founders, or hackers.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#137

Post by woadwarrior01 »

sanchitmonga22 wrote: post #105
woadwarrior01 wrote: post #93

> Apple M3 or later required. MetalRT uses Metal 3.1 GPU features available on M3, M3 Pro, M3 Max, M4, and later chips. M1/M2 support is coming soon. On M1/M2, RCLI automatically falls back to the open-source llama.cpp engine. So, no support for M5 Neural Accelerators, eh? (Requires Metal 4) ¯\_(ツ)_/¯

Ha, not yet. Metal 4 is interesting and we're keeping an eye on it. MetalRT currently targets Metal 3.1 GPU compute because that's where we get the most control over the decode pipeline. Neural Engine / ANE is powerful for fixed-shape inference (vision, classification) but autoregressive LLM decode, where you're generating one token at a time with dynamic KV cache, doesn't map as cleanly to ANE today. That said, if M…

> Neural Engine / ANE is powerful for fixed-shape inference (vision, classification) but autoregressive LLM decode, where you're generating one token at a time with dynamic KV cache, doesn't map as cleanly to ANE today.

What does the ANE have to with this?

Neural Engine (ANE) and the M5 Neural Accelerator (NAX) are not the same thing. NAX can accelerate LLM prefill quite dramatically, although autoregressive decoding remains memory bandwidth bound.

I suspect the biggest blocker for Metal 4 adoption is the macOS Tahoe 26 requirement.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#138

Post by concats »

shubham2802 wrote: post #76
stingraycharles wrote: post #4

I’m a bit confused by what you’re offering. Is it a voice assistant / AI as described on your GitHub? Or is it more general purpose / LLM ? How does the RAG fit in, a voice-to-RAG seems a bit random as a feature? I don’t mean to come across as dismissive, I’m genuinely confused as to what you’re offering.

RunAnywhere builds software that makes AI models run fast locally on devices instead of sending requests to the cloud. Right now, our focus is Apple Silicon. Today there are two parts: MetalRT - our proprietary inference engine for Apple Silicon. It speeds up local LLM, speech-to-text, and text-to-speech workloads. We’re expanding model coverage over time, with more modalities and broader support coming next. RCLI -…

How does it compare for models of any meaningful size?

These 0.6B-4B models are, frankly, just amusing curiosities. But commonly regarded as too error prone for any non-demo work.

The reason why people are buying Apple Silicon today is because the unified memory allows them to run larger models that are cost prohibitive to run otherwise (usually requiring Nvidia server GPUs). It would be much more interesting to see benchmarks for things like Qwen3.5-122B-A10B, GLM-5, or any dense model is the 20b+ range. Thanks.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#139

Post by mips_avatar »

sanchitmonga22 wrote: post #103
mips_avatar wrote: post #73

Have you tried any really big models on a mac studio? I'm wondering what latency is like for big qwens if there's enough memory.

Not yet with MetalRT, right now we support models up to ~4B parameters (Qwen3 4B, Llama 3.2 3B, LFM2.5 1.2B). These are optimized for the voice pipeline use case where decode speed and latency matter more then model size. Expanding to larger models (7B, 14B, 32B) on machines with more unified memory is on the roadmap. The Mac Studio with 192GB would be an interesting target, a 32B model at 4-bit would fit comfortably…

Well it’s just more that I’ve noticed in the agents I’ve built that qwen doesn’t get reliable until around 27b so unless you want to rl small qwen I don’t think I would get much useful help out of it.
Post reply on HN