Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon costs more than OpenRouter

williamangel.net

Post reply on HN
Print view View original

81–90 of 322 posts

Re: Apple Silicon costs more than OpenRouter

#81

Post by statestreet123 »

bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

Rounded up, yes, and oddly inefficient for someone obsessed with inefficiency. One could buy a brand new 64gb M5 macbook for well over 4k. Another could buy a scratched up but functioning M1 Max 64gb off of ebay for a little over 1k—and somehow get the same 10-20 t/s with 31b that the author does with an M5. Or better yet, have a frontier model do the planning and judging, and have a local MOE model execute at 50 t/s. All of this achievable by a former English major with too much free time.

Re: Apple Silicon costs more than OpenRouter

#82

Post by dijit »

vlovich123 wrote: post #75
dijit wrote: post #71

Earlier quoted context omitted.

Profitable for inference if you completely ignore training costs and that you absolutely must continuously train new models.

Which is where your analogy breaks down and why you think you’re taking crazy pills. Inference is growing and selling the oranges in your analogy. Model building is growing the farm to sell larger, juicier more addicting oranges.

Are ya fuckin' serious mate?

The restaurant next to the mines were profitable up until the moment the mines themselves shut down: one doesn't exist without the other.

You can't ringfence inference as "the profitable bit" and then hand-wave away the training. Without continuous training there is no inference product.

Claude 3 Opus isn't sitting there making revenue in 2026 - the thing is just deprecated. The moment you stop spending billions on the next model, your "profitable" inference business is on borrowed time until someone else makes it obsolete.

Maybe I made a mistake in my analogy... They're not growing a farm and then selling oranges. They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running.

Re: Apple Silicon costs more than OpenRouter

#83

Post by brianwawok »

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

So many more efficiencies possible at scale though. I cannot keep a local model 98% utilized 24/7, at least not with my current workload. A big cloud can. I can’t power my servers with DC, I have this AC to DV conversion nonsense. The list goes on.

Re: Apple Silicon costs more than OpenRouter

#84

Post by miltonlost »

ianberdin wrote: post #69
dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

Do you have a proof? Anthropic’s CEO said they Are profitable. Same with OpenAI.

If only they had their books open to do more than just "say"

Re: Apple Silicon costs more than OpenRouter

#85

Post by brianwawok »

MuffinFlavored wrote: post #66
dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

> Frontier AI companies are selling at a loss. How big/deep of a loss? I feel like I read this every day for years that Uber did this same "idiotic, losing" strategy (how it was pitched/discussed) and then one day we woke up and... without much fuss, boom, they were profitable seemingly overnight.

Well and uber cut the driver pay in half and doubled the price. They didn’t really find any efficiencies, robo drivers don’t exist yet. Also why I hardly touch them anymore.

Re: Apple Silicon costs more than OpenRouter

#86

Post by trollbridge »

cyanydeez wrote: post #38
bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

nothing about the current data center craze looks efficient.

Probably because lots of data centres are being built (or half-built) which are sitting idle.

Re: Apple Silicon costs more than OpenRouter

#87

Post by tempest_ »

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

It is the model training that is dragging them down.

If the arms race stopped tomorrow the current price pays for the inference.

Re: Apple Silicon costs more than OpenRouter

#88

Post by vlovich123 »

dijit wrote: post #77
vanviegen wrote: post #65

Earlier quoted context omitted.

That seems unlikely. There are many providers for open models on openrouter. It seems unlikely that they are throwing money away for each token they sell. Also, there a good technical reasons for inference being much more efficient at scale.

The providers on OpenRouter serving open models aren't "throwing money away", agreed. But that's not the point I'm making. (or, it kind of is, but it's more high level than that). They're running spot and preemptible GPU instances (60-80% cheaper than on-demand), paying wholesale industrial electricity rates, and running at multi-tenant utilisation densities that make your MacBook look like a bonfire. Of course they'…

> The floor price you're seeing isn't a stable equilibrium, it's a pile of investor money that hasn't entirely finished burning yet.

All that says is that it gets more expensive in the future as competitors exit the market and sustainability becomes important. That’s why Uber and Lyft were so cheap until they killed taxis. One major difference of course is that some models will remain largely good enough and the incremental cost of running will keep dropping to 0 over time since the hardware needed doesn’t get more expensive and is already purchased.

Re: Apple Silicon costs more than OpenRouter

#89

Post by Archit3ch »

Except I already have a local Mac to run Xcode. OpenRouter cannot help with that, at any price.

> 64 gigs should run a model like Gemma 4 31b

No, it can run anything in the 70B range. It's a notable quality upgrade from the 30B, which isn't obvious because the famous flurry of April releases didn't contain any 70Bs.

It can also run 120B in UD-Q3. Or 230B disk-streamed.

Post reply on HN