Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon costs more than OpenRouter

williamangel.net

Post reply on HN
Print view View original

171–180 of 322 posts

Re: Apple Silicon costs more than OpenRouter

#171

Post by apf6 »

bastawhiz wrote: post #55
dist-epoch wrote: post #28

Earlier quoted context omitted.

using it 24/7 brings the average cost down, not up. the less you use local LLM, the less sense it makes since you paid a lot for hardware you don't use

That's the point: why would you buy a device that's specifically not optimized to be used for 24/7 inference? It's expensive hardware that's not designed to be used in that situation! The power use for inference isn't especially good and you're not getting even a fraction of the benefit from the hardware that you're paying for.

Good question but people are doing it anyway. It's a fact that right now tons of people are buying Mac Minis specifically for this use case, to treat them as their personal data center for agents. The concept of "power use for inference" is foreign. Those people are the ones that motivated this blog post I think.

Re: Apple Silicon costs more than OpenRouter

#172

Post by atq2119 »

dijit wrote: post #82
vlovich123 wrote: post #75

Earlier quoted context omitted.

Which is where your analogy breaks down and why you think you’re taking crazy pills. Inference is growing and selling the oranges in your analogy. Model building is growing the farm to sell larger, juicier more addicting oranges.

Are ya fuckin' serious mate? The restaurant next to the mines were profitable up until the moment the mines themselves shut down: one doesn't exist without the other. You can't ringfence inference as "the profitable bit" and then hand-wave away the training. Without continuous training there is no inference product. Claude 3 Opus isn't sitting there making revenue in 2026 - the thing is just deprecated . The moment y…

> Without continuous training there is no inference product.

This claim deserves teasing apart.

Clearly, training is a Red Queen's race today. If a model provider were to unilaterally decide to stop training, they would very quickly lose market share to competitors with better models.

On the other hand, what if market and investment conditions change such that everybody has to stop training?

In that case, the models are still there and still as useful as they were the day before. So why wouldn't there still be an inference product?

Re: Apple Silicon costs more than OpenRouter

#173

Post by amluto »

maho wrote: post #9

The author only compared output token costs -- but for typical agentic workloads, input tokens dominate the costs by a large margin. Running inference locally, input tokens are, to first order, free. (They only generate implicit costs through higher time-to-first-token, higher power use, and lower token output speed).

Even ignoring superior caching on a local setup, Mac hardware can often process input token around 10x as quickly as they produce output tokens. Openrouter seems to have only a 2x difference on the same models.

Re: Apple Silicon costs more than OpenRouter

#174

Post by atq2119 »

Danox wrote: post #99
tempest_ wrote: post #87

Earlier quoted context omitted.

It is the model training that is dragging them down. If the arms race stopped tomorrow the current price pays for the inference.

But isn’t training models, a forever task like iterating in tech you can never take a day off, adding humans to the equation don’t humans train/teach themselves new skills over a lifetime, and isn’t one of the selling points in the future when selling this AI slop your AI never goes to sleep and can always be trained forever? The AI price for entry as we go on into the future will only increase.

I agree that training is a forever task, and the current rate of training is probably not sustainable. But all that means is that once the current investment mania ends, the market will most likely find a new equilibrium where continuous training still happens, but at a slower rate that can be sustained by inference revenue.

Re: Apple Silicon costs more than OpenRouter

#175

Post by Anon1096 »

bjt wrote: post #113
vlovich123 wrote: post #93

Earlier quoted context omitted.

> They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running. You’re literally describing all companies. Google takes about $270bn/year to run. If they stopped spending that they’d die pretty darn quick. It’s also a description of working - unless you’d built up significant savings, if you stopped working you’re also going to die.

> You’re literally describing all companies. No, not quite. It really comes down to opex vs capex and the depreciation schedule for your investment. Software development is typically categorized as capex, on a 3-5 year depreciation schedule. You assume the software you write today will be generating value for you that long. If a big, expensive model training project only gives you value for a year or less, that is no…

Software that is sold as a service and requires ongoing maintenance like running in the cloud (and people to keep it running in the cloud) is opex not capex. Google Search is most definitely opex.

Re: Apple Silicon costs more than OpenRouter

#176

Post by sgt »

ericpauley wrote: post #152
odo1242 wrote: post #150

Earlier quoted context omitted.

Yea this; it’s the same reason why mortgaging is cheaper than renting

This is far from a universal truth: https://www.nytimes.com/interactive/2024/upshot/buy-rent-cal... Real estate is only a clearly good investment if you ignore opportunity cost.

Articles like that still miss a bit of the nuance. Imagine having your house paid for, and you grow old and you have no rent to pay. Yes, you could have invested but likely you would have spent some of that money on something else, or your investments might have not worked out so well, or any other reason. Human reasons, to be specific. Owning property is like a lock.

Re: Apple Silicon costs more than OpenRouter

#177

Post by rprend »

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

This is not true. API tokens are not sold at a loss, and hardware gets more efficient over time, so serving inference on the same model gets cheaper. LLAMA 3.1 405B parameters was $6/$12/M tokens in 2024, but in 2026 that same model is $3/$3/M tokens.

The most intelligent model at a given time is much larger than the previous, which is why token costs for GPT5.5 are higher than 5.4. But you should expect that 2 years from now, serving a GPT5.5 sized model will be cheaper than GPT5.5 today. You should expect it to be even cheaper to get an equally intelligent model 2 years from now, because distillation techniques are effective at reducing the necessary parameter count for the same benchmark scores.

Re: Apple Silicon costs more than OpenRouter

#178

Post by empath75 »

It should not at all be surprising that running models at home is more expensive than commodity providers. That's just generally true of running your own stuff. Even if the cost in money isn't higher, the cost in time is often _significantly_ higher.

This is why the idea that the AI labs are in trouble because inference will be a commodity is _completely backwards_. Some of the largest and most powerful companies in the world sell commodities. They compete on scale and efficiency, and you are never going to be able to compete with the big labs on either.

Re: Apple Silicon costs more than OpenRouter

#179

Post by znpy »

I think that the main flaw in the reasoning is assuming that cost of token will stay the same over the years.

Chances are that token prices will go down, but chances also are that the AI bubble pops and all of a sudden all these companies will either have to make a buck out of the inference or go bankrupt.

Getting your own hardware just grants you stable pricing.

Re: Apple Silicon costs more than OpenRouter

#180

Post by mantas »

seanmcdirmid wrote: post #166
ericpauley wrote: post #152

Earlier quoted context omitted.

This is far from a universal truth: https://www.nytimes.com/interactive/2024/upshot/buy-rent-cal... Real estate is only a clearly good investment if you ignore opportunity cost.

You also need to pay close attention to rent vs purchase ratios. A lot of cities are cheap to rent but expensive to buy (eg beijing 10 years ago).

Key word being „ago“.
Post reply on HN