Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

github.com

Post reply on HN
Print view View original

111–120 of 167 posts

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#111

Post by sanchitmonga22 »

alfanick wrote: post #9

I'm not looking for STT->AI ->TTS, I'm looking for truly good voice-to-text experience* on Linux (and others). Siri/iOS-Dictation is truly good when it comes to understanding the speech. Something this level on Linux (and others) would be great, yeah always listening, maybe sending the data somewhere, but give me UX - hidden latency, optimizing for first chars recognized - a good (virtual) input device.

Understood, you want dictation, not a chatbot. That's a valid and different use case.

RCLI is Apple Silicon only today because MetalRT is built on Metal. For Linux, the closest thing to what you're describing would be building a virtual input device on top of Whisper or Parakeet (which RCLI supports as STT backends). Parakeet TDT 0.6B has ~1.9% WER, that's very close to production dictation quality.

The missing piece on Linux isn't the model, it's the integration: a daemon that captures mic audio, runs STT with hidden latency (streaming partial results), and injects text as keyboard input. sherpa-onnx (https://github.com/k2-fsa/sherpa-onnx) supports Linux and has streaming STT, it might be the best starting point for what your after.

We're focused on Apple Silicon for now but broader platform support is on the roadmap.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#112

Post by sanchitmonga22 »

jonhohle wrote: post #26

If I send a Portfile patch, would you consider MacPorts distribution?

Absolutely, we'd welcome a Portfile contribution. Happy to review and merge. If halostatue wants to co-maintain, even better.

Feel free to open a PR or issue on the RCLI repo and we'll coordinate.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#113

Post by sanchitmonga22 »

focusgroup0 wrote: post #32

The fact that Apple didn't ship this in years after Siri acquisition is an indictment of its Product leadership

Apple has the silicon, the frameworks (MLX, CoreML), and the models. The gap is putting it all together into a fast, unified on-device pipeline. That's what we're focused on, and honestly, we think Apple will eventually ship something similar natively. Until then, we're trying to show whats possible today on their hardware.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#114

Post by sanchitmonga22 »

computerex wrote: post #19

Amazing, this is what I am trying to do with https://github.com/computerex/dlgo

Cool, just checked out dlgo. Looks like you're targeting Go bindings for on-device inference? Different approach but same conviction that this should run locally. Happy to compare notes if you want to chat about Metal optimization or pipeline architecture.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#116

Post by neya »

shubham2802 wrote: post #96
mnafees wrote: post #91

Seems like you are leaking an ElevenLabs API key in your web demo. The OpenAI completions endpoint also has the API key in the request header but that seems to already be revoked and is returning a 401.

I am pretty sure we don't have balance. It's a bait :)

Sorry, but, this is not really a confidence inspiring response. Accepting the mistake and fixing the leak altogether would have been the better way to handle this. This is a developer forum, we all make mistakes. Framing it as bait just sounds like bad PR management.

How can we trust your product if you can't fulfil basic security 101? Not being harsh but this kind of lax response for a serious mistake is not acceptable to me. Imagine I recommend you to my company and you end up leaking out our credentials and respond with something like this.

I might be picky here about this, but long term trust starts with accountability.

All the best on your product launch and cheers.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#117

Post by [deleted] »

wtallis wrote: post #85
tomhow wrote: post #82

Earlier quoted context omitted.

This post is about the launch of a YC company and its product. It's reasonable that when a company launches a product, the discussion is focused on the product that it is launching. We moderate that way whether it's a YC company's Launch HN or anyone else posting a Show HN. Keeping discussion on-topic is one of the most important things we moderators do, and is the main reason HN is a place where people like to parti…

The idea of giving spammers a second chance seems truly bizarre to me. Have you ever un-blocked an email address that you previously blacklisted for spamming you? Do you think recipients of spam from this company want to give them a second chance? I'm not necessarily saying the people behind this should be completely blacklisted from the entire industry, but when a company earns a place on my block list for behavior…

[deleted]

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#118

Post by sanchitmonga22 »

emmelaich wrote: post #97
Tacite wrote: post #48

Earlier quoted context omitted.

This option didn't work either. I tried it. Also, the install script… installs Brew. So at the end, it's the same?

Oh dear. if ! command -v brew &>/dev/null; then info "Installing Homebrew..." /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" eval "$(/opt/homebrew/bin/brew shellenv)" fi

Fair point. The install script shouldn't silently install Homebrew without explicit consent. We'll update it to detect when Homebrew is missing and prompt the user before installing anything beyond RCLI itself.

In the meantime, if you already have Homebrew, you can install directly:

brew tap RunanywhereAI/rcli https://github.com/RunanywhereAI/RCLI.git brew install rcli rcli setup

Or build from source if you prefer not to use either method: https://github.com/RunanywhereAI/RCLI#build-from-source

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#119

Post by sanchitmonga22 »

antipaul wrote: post #52
pzo wrote: post #51

FWIW this RCLI is only MIT license but their engine MetalRT is commercial. Not sure the license of their models I guess also not MIT. So IMHO this repo is misleading. Not sure why they decided to reinvent the wheel and write yet another ML engine (MetalRT) which is proprietary. I would most likely bet on CoreML since it have support for ANE (apple NPU) or MLX. Other popular repos for such tasks I would recommend: htt…

Nice list. What about for on-device RAG use cases?

RCLI includes local RAG out of the box. You can ingest PDFs, DOCX, and plain text, then query by voice or text:

rcli rag ingest ~/Documents/notes rcli ask --rag ~/Library/RCLI/index "summarize the project plan"

It uses hybrid retrieval (vector + BM25 with Reciprocal Rank Fusion) and runs at ~4ms over 5K+ chunks. Embeddings are computed locally with Snowflake Arctic, so nothing leaves you're machine.

Re: Launch HN: RunAnywhere (YC W26) – Faster AI Inference on Apple Silicon

#120

Post by sanchitmonga22 »

Tacite wrote: post #37
esafak wrote: post #10

Earlier quoted context omitted.

You could share your setup details, on GH if not here, to make it actionable.

I did on Github. This looks vibecoded? EDIT: Dev is using Claude Code as stated in their github updates.

We use AI tools in our workflow, same as a lot of teams at this point. The pipeline architecture, Metal integration, and engine design are ours. The code is MIT and open for anyone to read and judge the quality directly.
Post reply on HN