21–30 of 436 posts
This is basically bunk because AI costs have gone down by 50x or more (api costs) since 3 years.
This is basically bunk because AI costs have gone down by 50x or more (api costs) since 3 years.
For awhile it was every 2-3 years you'd start a hardware refresh. As companies moved into more and more training, this timeframe started to shrink. It went from 36 months to 24 months. From 24 months to around 16-18 months. Last I checked last year, it was at 12 months. I think things may have slowed because of component availability, but otherwise whole data centers would be 6-12 months into full operations before they would start a refresh cycle.
Not to mention the massive increase in power density demand and cooling demand per rack that entails.
So no, "AI costs" have not gone down, in fact they are more expensive on training AND inference than ever.
This is why many are concerned about the heroin drip of api costs into orgs. For the companies that are public, look into their financials. It's gonna hit companies and high volume users like a ton of bricks.
Spelling mistake: "a return on these invetment"
I didn't get the sense this was LLM-written, but typo-signalling is... I donno a bit weird. Firefox is underlining some of the words as I write. I'm leaving "donno" unchanged even though it's flagging it as a misspelling but I suppose I'd still opt to fix something like "maiinstream" even at the risk of potentially seeming more LLM-ish!
"Crisis"
The article fails to mention DeepSeek, Alibaba, Qwen, Xiaomi, MiMo, z.ai, or GLM. It's hard to take such an article seriously that doesn't do this. (Our monthly total spend is around $180 with a team of 6, about half technical; our biggest line items are for American models or subscriptions which we probably will be planning to get rid of.) And then remarks like this: Anthropic, OpenAI and Microsoft have all now tran…
I really can’t stand when writers point to the difference in price per token on the api and subscription and use that as evidence that inference loses money. This author even says it’s implausible that the api charges 4x marginal cost when I think it’s very likely even higher than that. The entire rest of the post sits on this faulty assumption. Fixed costs don’t matter when marginal revenue is profitable and growing…
Do these knowledge jobs have a significant corpus of not only knowledge but discussion and problem solving, all conveniently labelled for the AI to train on? Probably not. Coding has stack overflow, what does, say, advertising use?
Chinese models and open model providers are, indeed, competing on price, and the difference shows.
Shouldn't we know a better answer to these questions once Anthropic's IPO materials surface publicly? I understand, and maybe even expect, SpaceX's materials to be all over the place and skate on by any discussion of unit economics, but the nerds over at Anthropic might just be forthright enough to just tell us what their margin is on tokens as part of their IPO.
Well it probably doesn't help that Dario is going around on podcasts saying things like "frontier labs need $1T of revenue or they will go bankrupt" lol.
I don't have a crystal ball, but based on similar historical scenarios, I think that one or two of these companies will win--probably because of some unique application, delivery or trade secret that will drive 80% of their revenue. Consider Google, Apple, Amazon, etc. It's still early days...
Having growth up in the 90s, it is weird seeing companies share their technology secrets publicly.
The article fails to mention DeepSeek, Alibaba, Qwen, Xiaomi, MiMo, z.ai, or GLM. It's hard to take such an article seriously that doesn't do this. (Our monthly total spend is around $180 with a team of 6, about half technical; our biggest line items are for American models or subscriptions which we probably will be planning to get rid of.) And then remarks like this: Anthropic, OpenAI and Microsoft have all now tran…
> Our monthly total spend is around $180 with a team of 6, about half technical; our biggest line items are for American models or subscriptions which we probably will be planning to get rid of.) Please tell more :). Do you pay per token from bedrock / openrouter / somewhere else? How many tokens you use over the month, and how many for each task? Which harnesses?