271–280 of 426 posts
Earlier quoted context omitted.
I print documents and photos at home regularly but I still contract out to dedicated print shops. The print shop can’t replicate the practicality of local printing and I can’t replicate their scale of investment. Both coexist perfectly.
Print-outs are a physical good. Tokens aren't.
Earlier quoted context omitted.
GPT-4 (original API): Input: $30 / 1M tokens Output: $60 / 1M tokens GPT-5.5: Input: $5 / 1M tokens Output: $30 / 1M tokens Costs have been reducing by over 5x year over year. Inference cost concern is mostly performative. https://simianwords.bearblog.dev/conclusive-proofs-that-llm-... Edit: can't reply but companies aren't selling inference at loss. In the blog post I point to third party hosting of open models like…
What they charge people says nothing about what it costs them. Off the top of my head, one confounding factor is trying to win back marketshare from Anthropic. We will only know the actually situation once Anthropic goes public and we can look at their books.
Earlier quoted context omitted.
Have you looked into DwarfStar 4?
Been away from home for nearly a month, so was mostly going off Qwen 3.5 122b-a10b (Q4?) / Qwen 3.6 35b-a3b (Q8) / Gemma4 31b (Q8) Thanks for suggestion tho, tool by antirez is always going to pique interest, I'll check it out when I'm finally home again Tho says Metal / CUDA, so doesn't seem friendly to Linux AMD system
Earlier quoted context omitted.
> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.
> Based on what? I take it you haven’t actually run any of the current gen local models? They all fit on fairly accessibility hardware, and their performance is at least on par with what I was paying for last year. I have one of my agents running entirely from a local model running on a MBP and it has repeatedly shown it’s capable of non-trivial tasks. Playing around with another, uncensored, local model on my 4090 d…
Earlier quoted context omitted.
Local modals are 6 months to 18 months behind frontier. Even if the performance of a cloud model is faster, it's clear that local is catching up.
> Local modals are 6 months to 18 months behind frontier. I wish this was true but it is not. And I am working on open source models so if anything, I would have a bias towards agreeing with you. Frontier closed models (GPT/Claude) are gaining distance to everybody else. Even Google, once the king. Your claim is a meme coming from benchmark results and sadly a lot of models are benchmaxxed. Llama 4, and most notably…
Gemma 4 was a major improvement is self-hostable local models and Qwen-3.6-A34B is a beast, and runs great on an MBP (and insanely well on a 4090).
The biggest lift is combining these models with a good agent harness (personally prefer Hermes agent). But I’ve found in practice they’re really not benchmaxxing. I’ve had these agents successfully hand a few non-trivial research projects that I wouldn’t have been able to accomplish as successfully even last year.
When you add in the open-but-not local models, Kimi, GLM, Minimax, you have a lot of very nice options. For personal use anything I don’t use local models for I give to my Kimi 2.6 powered agent.
Earlier quoted context omitted.
What they charge people says nothing about what it costs them. Off the top of my head, one confounding factor is trying to win back marketshare from Anthropic. We will only know the actually situation once Anthropic goes public and we can look at their books.
I think it's pretty safe to assume they are not losing money on inference.
Earlier quoted context omitted.
Tokens can be sold at profit, but 70% of compute expenditure goes to R&D and model training[0]. Inference needs to cover all of that as well as being profitable in a vacuum. [0] https://epoch.ai/data-insights/openai-compute-spend
this will change as inference demand increases (which is happening right now faster than many people expected)
Inference compute is definitely scaling fast, but to scale RL, training and R&D compute also needs to scale hard. I don't think it's obvious that inference will overtake R&D/training, unless there's a reputable source that states that.
[flagged]
"It costs OpenAI less money to serve GPT-5.5 than GPT-4." does it though? do you have the numbers? Or you just making stuff up?
Earlier quoted context omitted.
What they charge people says nothing about what it costs them. Off the top of my head, one confounding factor is trying to win back marketshare from Anthropic. We will only know the actually situation once Anthropic goes public and we can look at their books.
I think it's pretty safe to assume they are not losing money on inference.
Earlier quoted context omitted.
"It costs OpenAI less money to serve GPT-5.5 than GPT-4." does it though? do you have the numbers? Or you just making stuff up?
GPT-4 (original API): Input: $30 / 1M tokens Output: $60 / 1M tokens GPT-5.5: Input: $5 / 1M tokens Output: $30 / 1M tokens Costs have been reducing by over 5x year over year. Inference cost concern is mostly performative. https://simianwords.bearblog.dev/conclusive-proofs-that-llm-... Edit: can't reply but companies aren't selling inference at loss. In the blog post I point to third party hosting of open models like…
GPT-4.1 Input: $2.00 / 1M Tokens Output: $8.00 / 1M Tokens