Curren prices will come down. There is a lot of potential for optimization. Energy efficiency, energy generation, self hosting, model size and specialization. Etc. Rught now the state of the art is powering data centers with gas powered turbine generators. That's not very efficient.
Why current LLM costs are not sustainable
21–30 of 216 posts
Re: Why current LLM costs are not sustainable
#22Would prefer not to offend the author, but I do believe this article has very little for the HN audience. No new insight, and no numbers or new information.
Re: Why current LLM costs are not sustainable
#23> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…
The subscriptions will probably disappear or end up only allowed to use gimped versions of the models long term.
Re: Why current LLM costs are not sustainable
#24> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…
Why do you think that subscriptions are subsidized and not that enterprise tokens are sold at 3000% margin? There are few enough frontier labs that cartel is possible.
I think this comes from the idea that serving these tokens without paying for training is already expensive, e.g. https://news.ycombinator.com/item?id=46613887 self-hosted solution might give you only 10-100x more affordable solution at cost.
So, given the SOTA providers with even larger models also need to continously be using considerable resources for training their next models, to fund future data centers, and make profit, the token costs are more likely reflecting the real costs, rather than the subscription costs.
Re: Why current LLM costs are not sustainable
#25There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x
I am not sure if that is wise. It’s a hostile superpower after all
Which one? China or the US?
Re: Why current LLM costs are not sustainable
#26Earlier quoted context omitted.
Well... Open weights on premise is politically neutral.
Try doing it at scale for a whole office. Not trivial.
There are plenty of US based hosters racing to optimize and drive efficiencies
Literal race on twitter posting to increase token throughput and drive down costs on these Chinese open source models
Re: Why current LLM costs are not sustainable
#27> We are seeing improvements with each model release these days but it’s clear that the improvements are getting smaller and smaller. This is obviously untrue, both with GPT-5.4, and Claude Fable as examples in the last 6 months.
I would struggle to ascertain the day-to-day difference between GPT-5.4 and GPT-5.5 tbh. Also, imho, Fable is highly hyped, I don't think it is dramatically better than Opus 4.8. Maybe my tasks and interaction with AI is relatively simple (i.e., lots of Rust programming, Linux system engineering stuff).
Re: Why current LLM costs are not sustainable
#28[flagged]
Re: Why current LLM costs are not sustainable
#29There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x
I am not sure if that is wise. It’s a hostile superpower after all
DeepSeek first and foremost is a business. Yes, being a business in China means risk of being ordered tomorrow to do something that is not in your best interests. But now we know the US is not immune to that level of government oversight either.
The difference is DeepSeek and other Chinese models are open weights.
Re: Why current LLM costs are not sustainable
#30> We are seeing improvements with each model release these days but it’s clear that the improvements are getting smaller and smaller. This is obviously untrue, both with GPT-5.4, and Claude Fable as examples in the last 6 months.
gpt 5.5 regularly wastes tokens on wrong commands, requires lots of handholding. I highly doubt there's substantial improvement