Apple Silicon costs more than OpenRouter

Post by nomel » Sun, May 17, 2026, 7:43 PM UTC

datadrivenangel wrote: post #128

mpyne wrote: post #102

Earlier quoted context omitted.

They're responding to the people doing things like buying the most expensive Mac they can find specifically to do local inference for their AI agents. Some do it to have control over their ability to use AI. Some do it because they think it will be cheaper to not have to pay a SaaS to generate tokens for them. But for those interested in the latter case, it seems like it's not actually cheaper after all, at least at…

It's worth paying a premium for the privacy (assuming that llama.cpp and ollama aren't sending my sessions back to the cloud regardless...), and for the concerns about not getting a surprise bill.

Post by filleduchaos » Sun, May 17, 2026, 7:46 PM UTC

atq2119 wrote: post #164

llm_nerd wrote: post #94

Earlier quoted context omitted.

Your post makes sense if you bought the hardware for other reasons, and maybe run models occasionally as a novelty. That isn't the case for many, though, and there is a whole social media space where people are hyping up the latest homebrew options for running models, believing it frees them from the yoke of big AI. Millions of people are buying big $ maxed-out hardware like the Mac Studios or DGX specifically to run…

Let's not get ahead of ourselves. Millions , really? I can believe there are a lot of enthusiasts doing this, but "millions" needs a citation.

Post by eikenberry » Sun, May 17, 2026, 7:55 PM UTC

rprend wrote: post #177

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

This is not true. API tokens are not sold at a loss, and hardware gets more efficient over time, so serving inference on the same model gets cheaper. LLAMA 3.1 405B parameters was $6/$12/M tokens in 2024, but in 2026 that same model is $3/$3/M tokens. The most intelligent model at a given time is much larger than the previous, which is why token costs for GPT5.5 are higher than 5.4. But you should expect that 2 years…

Post by eikenberry » Sun, May 17, 2026, 7:58 PM UTC

visarga wrote: post #101

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

> Frontier AI companies are selling at a loss. There are huge economies to be had by batching requests and using lots of RAM for MoE (sparse models). You can't achieve that efficiency at batch size 1 on a single node.

Post by PunchyHamster » Sun, May 17, 2026, 8:01 PM UTC

bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…