What would really elevate an article like this is if we could somehow quantify human brain’s equivalent outputs and compare the costs with local LLM and cloud LLMs.
The computer does most specific tasks better, faster, and cheaper than I do.
Apple Silicon costs more than OpenRouter
121–130 of 322 posts
Re: Apple Silicon costs more than OpenRouter
#122Now include the externalized cost in the U.S. of deploying ~100% of productive capital to build data centers instead of, for example, first-world transportation infrastructure, and tell me which one is cheaper
Why would I want to include that when determining the cost per token?
Re: Apple Silicon costs more than OpenRouter
#123Earlier quoted context omitted.
I think Apple really do care and know that Moore’s law is likely to position them as major winners in this race in 3-7 years time.
This. The M5’s massive speed up in refill is a good sign. Apple isn’t expecting wholesale adoption of on-device models this year or next. But all of their design and iteration suggests they see it coming.
Apple has been doing tons of on-device ML for years now, but it's primarily background magic like photo tagging.
Re: Apple Silicon costs more than OpenRouter
#124Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…
The Michael Scott AI Companies.
Re: Apple Silicon costs more than OpenRouter
#125Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive…
> Frontier AI companies are selling at a loss. How big/deep of a loss? I feel like I read this every day for years that Uber did this same "idiotic, losing" strategy (how it was pitched/discussed) and then one day we woke up and... without much fuss, boom, they were profitable seemingly overnight.
As long as you have slaves/sharecroppers, driving the people at the top of the pyramid at Uber they’re profitable and Uber makes money as long as you don’t care about the workers and as long as you can get around all of the regulations that are put on traditional cab companies if there are any left on the road.
For me nothing says low class like the Porsche dealer saying we can call Uber for you to take you home ridiculous… and it was a low class experience dirty car small never again ha ha ha…
Re: Apple Silicon costs more than OpenRouter
#126Now include the externalized cost in the U.S. of deploying ~100% of productive capital to build data centers instead of, for example, first-world transportation infrastructure, and tell me which one is cheaper
Why would I want to include that when determining the cost per token?
It’s part of the cost per token
Re: Apple Silicon costs more than OpenRouter
#127Re: Apple Silicon costs more than OpenRouter
#128Earlier quoted context omitted.
The article makes no sense. I can't use OpenRouter as a general purpose computing device. Why are we comparing a whole computer to a single purpose SaaS?
They're responding to the people doing things like buying the most expensive Mac they can find specifically to do local inference for their AI agents. Some do it to have control over their ability to use AI. Some do it because they think it will be cheaper to not have to pay a SaaS to generate tokens for them. But for those interested in the latter case, it seems like it's not actually cheaper after all, at least at…
It's worth paying a premium for the privacy (assuming that llama.cpp and ollama aren't sending my sessions back to the cloud regardless...), and for the concerns about not getting a surprise bill.
Re: Apple Silicon costs more than OpenRouter
#129Earlier quoted context omitted.
Rounding everything down in the most optimistic setting got me to $0.40 per million tokens, and openrouter has the same model at $.38/mtok.
What is it with AI SaaS naming themselves "openxyz" when there is 0% open about them?
They learnt from ooenai that naming yourself open-xyz doesn't actually require opening anything.
Re: Apple Silicon costs more than OpenRouter
#130Unless I'm misunderstanding, this is counting the entire laptop in the cost of generating tokens. The calculation seems to omit that, in addition to receiving LLM output, you have also received a laptop in exchange for your money. If you intend to put this machine in a dark corner and run it solely as a token-munching server, a laptop would be an exceptionally poor choice of technology for this purpose. But if you in…
> in addition to receiving LLM output, you have also received a laptop in exchange for your money
And, since it's a Mac, whenever you're ready to upgrade it'll still have a fairly decent resale value.