Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon costs more than OpenRouter

williamangel.net

Post reply on HN
Print view View original

241–250 of 322 posts

Re: Apple Silicon costs more than OpenRouter

#241

Post by stusmall »

nativeit wrote: post #145
datadrivenangel wrote: post #27

Earlier quoted context omitted.

Rounding everything down in the most optimistic setting got me to $0.40 per million tokens, and openrouter has the same model at $.38/mtok.

But once all that is done you still own a Mac in one case, and you don’t in the other, correct?

Not always. The calculations take its useful life expectancy as an input. If they estimate it correctly you have highly likelihood of it breaking, burning out or being woefully out of date by the end. At the 10 year window you are looking at losing support for security updates.

So if you are lucky you might end up with something that still runs but most folks won't find it particularly useful

Re: Apple Silicon costs more than OpenRouter

#242

Post by BoorishBears »

xienze wrote: post #218
BoorishBears wrote: post #187

Earlier quoted context omitted.

How is it going to have decent resale when you bought it for $15,000 over MSRP because of a time-limited LLM mass-psychosis? https://www.ebay.com/sch/i.html?_nkw=apple+mac+studio+m3+ult... I think you'd need to tinker quickly, realize anything with CUDA (other than the awful DGX Spark) is better for learning, the prefill is killing your ability to actually run models large enough to justify that RAM, and then cure yo…

Well obviously I'm not talking about buying over MSRP.

And I am.

Given those models aren't sold at MSRP anymore and are primarily being sought for inference, it's apropos of an article on the costs of inference on Apple Silicon.

Re: Apple Silicon costs more than OpenRouter

#243

Post by curt15 »

llm_nerd wrote: post #94
bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

Your post makes sense if you bought the hardware for other reasons, and maybe run models occasionally as a novelty. That isn't the case for many, though, and there is a whole social media space where people are hyping up the latest homebrew options for running models, believing it frees them from the yoke of big AI. Millions of people are buying big $ maxed-out hardware like the Mac Studios or DGX specifically to run…

> Millions of people are buying big $ maxed-out hardware like the Mac Studios or DGX specifically to run LLMs.

What's your source for this?

Re: Apple Silicon costs more than OpenRouter

#244

Post by 650REDHAIR »

PAndreew wrote: post #211
quietsegfault wrote: post #45

Earlier quoted context omitted.

It’s more than just data locality. OpenRouter is faster, no? I have an M4 pro, and anything but the smallest dumbest models are unusably slow for interactive use. I personally haven’t yet found a good use case for offline/non-interactive LLM work locally.

I’m running a local Whisper + Gemma 4 pipeline with a cheap USB mic to extract health related data and potential todos from ambient speech. It doesn’t have to be fast doesn’t have to be 100% correct because if it captures at least a few bits of interesting information that would otherwise go unnoticed it’s still a win.

I run whisper through openwebui to gemma4 moe and use kokoro TTS back to me.

I use a 5060ti 16gb and a minipc.

I tunnel in via Tailscale and access it with my phone or laptop from anywhere. It’s pretty good and will only get better as I optimize.

Re: Apple Silicon costs more than OpenRouter

#245

Post by jeffybefffy519 »

NicuCalcea wrote: post #68
dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

The blog compares the cost of running Gemma4 31b, which on OpenRouter is offered by small no-name inference providers, not by frontier AI companies. It seems like a fair comparison to me.

Interestingly if you look at the cost of Gemma3 (this is 12 months old, but demonstrates the authors point) on Vertex AI versus Gemini 2.5 pro, the cost per million tokens WAS very similar.

Re: Apple Silicon costs more than OpenRouter

#246

Post by bigyabai »

amluto wrote: post #173
maho wrote: post #9

The author only compared output token costs -- but for typical agentic workloads, input tokens dominate the costs by a large margin. Running inference locally, input tokens are, to first order, free. (They only generate implicit costs through higher time-to-first-token, higher power use, and lower token output speed).

Even ignoring superior caching on a local setup, Mac hardware can often process input token around 10x as quickly as they produce output tokens. Openrouter seems to have only a 2x difference on the same models.

For larger contexts (eg. 20,000+ token agent workflows), being 10x faster still isn't enough. You have to be close to ~100x faster at crunching contexts for it to feel like realtime.

Re: Apple Silicon costs more than OpenRouter

#247

Post by g-technology »

jwr wrote: post #138

> "run a model like Gemma 4 31b, which is almost anthropic sonnet levels of performance" I wish people stopped deluding themselves — I regularly try (and benchmark for my purposes) local models and they are NOWHERE near the huge models like Sonnet or Opus. Nowhere. Yes, you can sometimes get plausibly-looking output for simple tasks, but for anything even remotely requiring thinking there is simply no comparison. Loc…

It all depends on use case. A local fine tuned model on a very specific use case can definitely out perform a much bigger cloud model that doesn’t have the training on your use case. But, that requires looking at the ai models as a means to end and not a Swiss Army knife that can do it all.

Re: Apple Silicon costs more than OpenRouter

#248

Post by vlovich123 »

Anon1096 wrote: post #175
bjt wrote: post #113

Earlier quoted context omitted.

> You’re literally describing all companies. No, not quite. It really comes down to opex vs capex and the depreciation schedule for your investment. Software development is typically categorized as capex, on a 3-5 year depreciation schedule. You assume the software you write today will be generating value for you that long. If a big, expensive model training project only gives you value for a year or less, that is no…

Software that is sold as a service and requires ongoing maintenance like running in the cloud (and people to keep it running in the cloud) is opex not capex. Google Search is most definitely opex.

There was some accounting changes passed during the Biden administration’s funding bill that made it capex for a few years. It’s been rolled back in the Trump administration’s bill and I suspect no one will ever let it happen again.

Re: Apple Silicon costs more than OpenRouter

#249

Post by pyreko »

trollbridge wrote: post #91
sleepyeldrazi wrote: post #64

If you want a good dense model, use qwen3.6 27B instead, speed will be up, and if you don't take my word for it being smarter, take openrouter's prices of it against the bigger, slower and less memory-efficient gemma do the talking. If you want a faster model, go for qwen3.6 35B (or gemma 4 26B if gemma models perform better for your tasks). There is a reason why people (myself included) haven't shut up about those t…

Right. Qwen 3.6 45b (6 parameter) runs on a commodity 5090, which, if you're into video games, you probably already have one of. It is entirely usable for most code generation tasks. (Not all, but most.) Likewise, DeepSeek V4 Flash is quite accessible on local models, with DwarfStar 4 making it easy to run on a 96GB MacBook. There's nothing wrong with paying for inference, but local models bring up some pretty amazin…

Ah yes, most people who play any sort of video game own a $2000 graphics card :p

Re: Apple Silicon costs more than OpenRouter

#250

Post by jmalicki »

nativeit wrote: post #145
datadrivenangel wrote: post #27

Earlier quoted context omitted.

Rounding everything down in the most optimistic setting got me to $0.40 per million tokens, and openrouter has the same model at $.38/mtok.

But once all that is done you still own a Mac in one case, and you don’t in the other, correct?

Even at just the electricity cost openrouter will be both

1) Roughly break-even to a little bit cheaper per token cost 2) Much, much, faster

So the cost of the mac barely even matters, it's just an extra cost beyond.

Sure, data center providers can pay lower rates.

The point of this article is that LLMs at home really don't make a ton of sense, unless you are willing to pay through the nose for privacy. There is absolutely no cost saving to be had.

If you're looking at your own datacenter as a larger corporate client, that could change.

There are also some providers that will contractually keep your data private, like AWS Bedrock or parts of Google/Azure (I don't know their stack names).

AWS even has AWS Secret Region and AWS Top Secret Region if you want to use LLMs on classified data.

You have to value privacy at a roughly absurd level to not want to use LLMs run efficiently at scale by someone else. For the home user, just the extra efficiency produced by batching requests from a large number of users in a datacenter in a real win.

Some of these companies are even selling tokens below cost to get marketshare. If someone will sell you a service for a dollar bill or three quarters, why wouldn't you take the three quarters?

Post reply on HN