This is basically bunk because AI costs have gone down by 50x or more (api costs) since 3 years.
What?
He's saying output for 1M tokens on the latest models is $50 now when it used to be $2500.
AI's Affordability Crisis
31–40 of 436 posts
Re: AI's Affordability Crisis
#32The article fails to mention DeepSeek, Alibaba, Qwen, Xiaomi, MiMo, z.ai, or GLM. It's hard to take such an article seriously that doesn't do this. (Our monthly total spend is around $180 with a team of 6, about half technical; our biggest line items are for American models or subscriptions which we probably will be planning to get rid of.) And then remarks like this: Anthropic, OpenAI and Microsoft have all now tran…
I think the author is referring to enterprise customers. You aren't the "customer" in this case; you're the bait.
How do you know that the other models you are referring to aren't subsidized?
Re: AI's Affordability Crisis
#33I really can’t stand when writers point to the difference in price per token on the api and subscription and use that as evidence that inference loses money. This author even says it’s implausible that the api charges 4x marginal cost when I think it’s very likely even higher than that. The entire rest of the post sits on this faulty assumption. Fixed costs don’t matter when marginal revenue is profitable and growing…
Then what are the real costs?
Re: AI's Affordability Crisis
#34The unit economics might be just fine. We'll know more after IPO.
The drug dealer analogy has a darker side to it, however.
Once your dependent, they can drive up the price just because. It doesn't need to be for existential reasons.
Re: AI's Affordability Crisis
#35Re: AI's Affordability Crisis
#36I don't have a crystal ball, but based on similar historical scenarios, I think that one or two of these companies will win--probably because of some unique application, delivery or trade secret that will drive 80% of their revenue. Consider Google, Apple, Amazon, etc. It's still early days...
The US govt is going to ban foreign models and foreign providers, and frontier labs are still cooked, because US companies will RLwash Chinese models to try and get in on the captive market. The frontier labs have already lost the war for coding, their next play is custom models for specific domains... Anthropic Galen for biomedical research, Anthropic Locke for legal analysis, etc, and you won't see _ANY_ intermediate work on the model, you will put in query, maybe get some questions fired back during work, and get a "final report."
Eventually the frontier labs will try to cut out the middle man once these models prove themselves and start doing partnerships with big firms in the domains, so they can take a % of the profits in perpetuity rather than just taking a one time payment. For example, after Anthropic Galen, they'll do a partnership with Pfizer to generate Ozempic-Superjacked and take 20% royalties on global sales.
Re: AI's Affordability Crisis
#37The unit economics might be just fine. We'll know more after IPO. The drug dealer analogy has a darker side to it, however. Once your dependent, they can drive up the price just because . It doesn't need to be for existential reasons.
It's a really different market, though. New entrants can easily undercut them if they price too high
Re: AI's Affordability Crisis
#38Earlier quoted context omitted.
> Our monthly total spend is around $180 with a team of 6, about half technical; our biggest line items are for American models or subscriptions which we probably will be planning to get rid of.) Please tell more :). Do you pay per token from bedrock / openrouter / somewhere else? How many tokens you use over the month, and how many for each task? Which harnesses?
Not the GP, but I use Opus for planning, Deepseek for actual coding (implementing the plan) and GPT for review. GPT is inexhaustible on the $20/mo plan, Deepseek is dirt cheap (maybe $10/mo) and Claude is Claude.
GP is talking about API / token-based prices, that's why I asked.
Re: AI's Affordability Crisis
#39Earlier quoted context omitted.
He's saying output for 1M tokens on the latest models is $50 now when it used to be $2500.
so how are these labs going to recoup the insane training costs at those prices? even if there is still a fat margin leftover afterwards
They also have to continuously train, forever, to avoid model drift. It's not a one and done thing as far as I'm aware.
Re: AI's Affordability Crisis
#40Spelling mistake: "a return on these invetment"
It's Proof of (human) Work. Much more useful than having a sticker saying "Done by a Human".
Is deleting a letter after an LLM generated the article an insurmountable task? These quaint signals only screen out the lowest of effort slop writers. Better than absolutely nothing, but barely.
It does remind me of the time a chef told me when he puts lemon juice over a dish, he would intentionally not remove any seeds that went on it because it was a signal of quality. I wonder if future slop chefs will intentionally place seeds on dishes that came from a box...