Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon costs more than OpenRouter

williamangel.net

Post reply on HN
Print view View original

131–140 of 322 posts

Re: Apple Silicon costs more than OpenRouter

#131

Post by onesociety2022 »

brianwawok wrote: post #85
MuffinFlavored wrote: post #66

Earlier quoted context omitted.

> Frontier AI companies are selling at a loss. How big/deep of a loss? I feel like I read this every day for years that Uber did this same "idiotic, losing" strategy (how it was pitched/discussed) and then one day we woke up and... without much fuss, boom, they were profitable seemingly overnight.

Well and uber cut the driver pay in half and doubled the price. They didn’t really find any efficiencies, robo drivers don’t exist yet. Also why I hardly touch them anymore.

All that tells me is they did find an efficiency. If they didn’t, their driver supply would have dropped. Unlike the taxi business, Uber/Lyft can tap into otherwise dormant supply of drivers who already own a car but aren’t willing to spend all 40-60 hours a week driving a taxi. With Uber/Lyft, they can become part-time drivers (they have flexibility and they can use an asset they already own anyway). Is it worse for the full time taxi drivers who used to have the supply artificially constrained in the old medallion system? Yes, but does it also benefit others who want to do this as a flexible job, zero skills required other than driving, no boss to deal with, no job interviews, etc. Yes!

Re: Apple Silicon costs more than OpenRouter

#132

Post by mbgerring »

mbgerring wrote: post #126
tuwtuwtuwtuw wrote: post #122

Earlier quoted context omitted.

Why would I want to include that when determining the cost per token?

It’s part of the cost per token

Can’t reply to the reply here, but yes, you do pay for it with money. Absorbing all of America’s capital and construction labor capacity to build AI data centers, rather than, for example, reaching parity with every other developed country in transportation infrastructure, is a cost you pay every day in gas and time spent in traffic. More to the point, building AI datacenters without paying for modernizing the grid is raising electricity rates. Token costs aren’t just subsidized by VC money, they are subsidized by all of us because of idiotic policy choices. Lots of people have pointed out that the current price per token is heavily subsidized, and a fair analysis would account for that.

Re: Apple Silicon costs more than OpenRouter

#133

Post by xienze »

vlovich123 wrote: post #75
dijit wrote: post #71

Earlier quoted context omitted.

Profitable for inference if you completely ignore training costs and that you absolutely must continuously train new models.

Which is where your analogy breaks down and why you think you’re taking crazy pills. Inference is growing and selling the oranges in your analogy. Model building is growing the farm to sell larger, juicier more addicting oranges.

In this particular case, inference and training are intertwined. It might be one thing if Anthropic could get away with training a new model every five years and control costs that way. But they can't. Put another way, their inference has no value without continuous, very expensive training. Because consumers aren't purchasing based on price but capability, otherwise the Chinese models on OpenRouter would have buried OpenAI and Anthropic already.

Re: Apple Silicon costs more than OpenRouter

#135

Post by Danox »

konaraddi wrote: post #57

A lot of comments here are about the issues with the analysis in OP’s post but much of them are “a distinction without a difference” with respect to the broader conclusion. When we look at purely cost and performance (setting aside privacy) then it’s better for individual devs to pay for hosted then for self hosting. Employers are paying for tokens on the job and most devs are finding the $PREFERRED_PROVIDER’s $20/$1…

The model makers, mainframe dream of computer’s isn’t coming back no matter what OpenAI, Google, Anthropic or Microsoft want, there are too many smart tech barbarians at the gate that want in and they’re not going to be satisfied to go back to the computer terminal era.

Personal computers eliminated an earlier terminal era, and most if not all of those companies are gone except for IBM and a few stragglers and they are a shell of their former selves.

Re: Apple Silicon costs more than OpenRouter

#136

Post by tuwtuwtuwtuw »

mbgerring wrote: post #132
mbgerring wrote: post #126

Earlier quoted context omitted.

It’s part of the cost per token

Can’t reply to the reply here, but yes, you do pay for it with money. Absorbing all of America’s capital and construction labor capacity to build AI data centers, rather than, for example, reaching parity with every other developed country in transportation infrastructure, is a cost you pay every day in gas and time spent in traffic. More to the point, building AI datacenters without paying for modernizing the grid i…

I live in one of the developed countries, not in the US.

Re: Apple Silicon costs more than OpenRouter

#137

Post by sheepscreek »

faitswulff wrote: post #46
bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

The article makes no sense. I can't use OpenRouter as a general purpose computing device. Why are we comparing a whole computer to a single purpose SaaS?

No, that’s not the point. I think this is to help people who are thinking about getting a beefier Mac so they can run their LLMs on it too. Some in particular want a dedicated Mac Mini or Studio for this purpose. The breakdown, even if slightly flawed, offers a good insight into the economics of it.

For most people, they might be better off with OpenRouter models and providers supporting Zero Data Retention. On the cloud, that’s as good as it gets for privacy - your data is never retained beyond the life of the request.

Re: Apple Silicon costs more than OpenRouter

#138

Post by jwr »

> "run a model like Gemma 4 31b, which is almost anthropic sonnet levels of performance"

I wish people stopped deluding themselves — I regularly try (and benchmark for my purposes) local models and they are NOWHERE near the huge models like Sonnet or Opus. Nowhere. Yes, you can sometimes get plausibly-looking output for simple tasks, but for anything even remotely requiring thinking there is simply no comparison.

Local models are useful. I use them for spam filtering, and soon intend to use them for image tagging and OCR. But let's stop saying they can get us "anthropic sonnet levels of performance", because that's just not true.

Post reply on HN