Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Silicon costs more than OpenRouter

williamangel.net

Post reply on HN
Print view View original

51–60 of 322 posts

Re: Apple Silicon costs more than OpenRouter

#51

Post by throw1234567891 »

maxdo wrote: post #43

I'm even surprised people ignorantly talking about advantages of buying very expensive device , run it only sometimes and aiming to beat cloud vendors. If small model is great it will be hosted with good electricity cost and will be utilized 24/7. Isn't it 2+2 of economics ? CPU is a commodity, and we are still buying cpu and ram from vendors for same reason

Put a cost on sending your intellectual property to a saas provider who knows where. Half a problem when it is just your IP, hopefully not the IP of your clients. Maybe if one is building yet another html nobody really cares about.

Re: Apple Silicon costs more than OpenRouter

#53

Post by gizajob »

jmyeet wrote: post #48

I've dug into this previously for one simple reason: NVidia segments the market by capping VRAM and Apple silicon uses a shared memory model that could challenge that but it currently doesn't. And I really wonder if Apple realizes the potential of what they have or if they even care. So, for comparison, a 5090 has 32GB of VRAM and you can get one for ~$3000 maybe. To go beyond that memory with current generation (ie…

I think Apple really do care and know that Moore’s law is likely to position them as major winners in this race in 3-7 years time.

Re: Apple Silicon costs more than OpenRouter

#54

Post by kamranjon »

antirez wrote: post #41

Mmmm, nope if you do the smart thing. MacBook M5 max 128gb is a premium laptop at 6k, but with it you can do many things and is your good main driver for the day. Then, it can also run DeepSeek V4 flash and perform non trivial tasks locally, without censorship or limitations, even without an internet connection and on very privacy sensitive data. That's a good deal. If you buy 25k for a dual Mac Studio 512gb to aband…

Yea my m4 max with 128gb has ended up making a lot of sense for me. I do video editing, I train ml models, I run large open AI models, I do 3d modeling, rendering and cad work. I never do all of this 100% of the time, I’ll setup a ml training to run over night and check results in the morning, during work I’ll set it up as a server and run local models, on my own time I’ll edit video and work on 3d modeling. It’s an incredibly versatile machine - and all of this is done while keeping your data on your device and giving you full control over your workflows.

Re: Apple Silicon costs more than OpenRouter

#55

Post by bastawhiz »

dist-epoch wrote: post #28
bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

using it 24/7 brings the average cost down, not up. the less you use local LLM, the less sense it makes since you paid a lot for hardware you don't use

That's the point: why would you buy a device that's specifically not optimized to be used for 24/7 inference? It's expensive hardware that's not designed to be used in that situation! The power use for inference isn't especially good and you're not getting even a fraction of the benefit from the hardware that you're paying for.

Re: Apple Silicon costs more than OpenRouter

#56

Post by bastawhiz »

cyanydeez wrote: post #38
bastawhiz wrote: post #14

This isn't a good analysis, and it's because it keeps rounding everything up. He rounds up the cost of electricity by 10%. He has a range of power use, takes the high end (which is 2x the low end) and multiplies it by the inflated electricity cost. But then they talk about using a newly purchased Mac to do the inference, running at full capacity, 24/7. Why would you do that? Apple silicon is fast but the author point…

nothing about the current data center craze looks efficient.

Whether you think building data centers or not is a good idea it's inarguable that the per-token efficiency (power, hardware, etc) is FAR higher in a data center. That's literally what it's designed for.

Re: Apple Silicon costs more than OpenRouter

#57

Post by konaraddi »

A lot of comments here are about the issues with the analysis in OP’s post but much of them are “a distinction without a difference” with respect to the broader conclusion. When we look at purely cost and performance (setting aside privacy) then it’s better for individual devs to pay for hosted then for self hosting. Employers are paying for tokens on the job and most devs are finding the $PREFERRED_PROVIDER’s $20/$100/$200/month subscription sufficient outside of work. Most devs don’t fall in the conditions under which running local models make sense purely on the basis of cost vs performance.

More critically, in practice, setting up local models seems more like a hobby, an educational exercise, or an act of privacy control than it is for cost cutting or productivity.

Re: Apple Silicon costs more than OpenRouter

#58

Post by brookst »

gizajob wrote: post #53
jmyeet wrote: post #48

I've dug into this previously for one simple reason: NVidia segments the market by capping VRAM and Apple silicon uses a shared memory model that could challenge that but it currently doesn't. And I really wonder if Apple realizes the potential of what they have or if they even care. So, for comparison, a 5090 has 32GB of VRAM and you can get one for ~$3000 maybe. To go beyond that memory with current generation (ie…

I think Apple really do care and know that Moore’s law is likely to position them as major winners in this race in 3-7 years time.

This. The M5’s massive speed up in refill is a good sign.

Apple isn’t expecting wholesale adoption of on-device models this year or next. But all of their design and iteration suggests they see it coming.

Re: Apple Silicon costs more than OpenRouter

#59

Post by dijit »

Frontier AI companies are selling at a loss.

Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left.

If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive for you to do the growing.

I feel like I'm taking crazy pills. These models will become more expensive over time, it's functionally impossible for them not to, they just want to capture the market before they have to stop selling at a huge loss.

[0]: https://news.ycombinator.com/item?id=48168433

Re: Apple Silicon costs more than OpenRouter

#60

Post by poly2it »

dijit wrote: post #59

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…

Well, I'd be surprised if non-R&D inference providers were selling at a loss. There are a plethora to choose from, competition is quite healthy. Will they keep providing cheap tokens while the labs raise their prices? Probably, but then I don't see how they could be raised in the first place. And what timescale are you talking about? A couple of years? It is appropriate to assume inference will become more efficient over time. If you raise your prices, you are going to be out competed before it's profitable (if you assume it is unprofitable) which would be negligent. I don't see how this makes sense.
Post reply on HN