I would think it depends on what you count in as cost, how long you can operate a data center, in which intervals you need to train new frontier models etc.
There are lots of knobs to dial for your costs.
Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
21–30 of 89 posts
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#22If this is the true cost of AI then the future might be dedicated extension cards for computers that hardcode entire models + weights. Downside: you need to buy a new one for each model. Upside: insanely fast inference and zero subscription cost, only one time purchase cost. Once a certain open source model gets good enough this might become viable. Right now the landscape is still shifting too fast. State of the art…
Yes this is what I"m waiting for. I do hope these cards will not come with any kind of vendor locking though.
What kind of vendor locking would be possible?
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#23I really don't now how these other code bases are structured. Our team ran cc-usage and our cost is right about what we pay as our monthly license. This is only those on the team pro side.
Our code base is not small, millions of lines of code. It does not take $65 in tokens to solve an issue. I'm running 3-4 claude code terminals at the same time and i'm still pretty close to what it would cost per a token for usage. I don't know what we are doing right with our code or claude.md to make this happen and I don't want to change it to break it.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#24So we're in the $6 Uber ride era of AI still?
[deleted]
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#25Hang on to your wallet, you ain't seen nothing yet. The true cost of AI won't be revealed until after a large portion of the customer base has become "hooked" on it.
“Hooked” is doing a lot of work there.
I’m happily paying $200/mo for Claude code. The tokens I use would be >$10k at API rates. I’m building the best products of my life, and multiple of them in parallel. I’m doing better creative work, finally realizing a game idea I’ve had no time for, etc.
If this level of usage goes to $500/mo… I’ll be out. It’s worth what I’m paying, but hey I went a decade without writing that song or building that game. It’s not freaking heroin, it’s a tool that offers a good value for what I pay.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#26If this is the true cost of AI then the future might be dedicated extension cards for computers that hardcode entire models + weights. Downside: you need to buy a new one for each model. Upside: insanely fast inference and zero subscription cost, only one time purchase cost. Once a certain open source model gets good enough this might become viable. Right now the landscape is still shifting too fast. State of the art…
I would shell out cash right now for something like Opus on silicon, like what Taalas [0] has built for Llama 3.1.
Having lightning-speed, local inference of a super high-quality model would be incredible. If you haven't played with it, check out Taalas's demo [1].
Honestly, though - I have my doubts. Recurring revenue is just too nice to pass up; I'm sure AI companies wouldn't want me buying a dedicated Opus card and not giving them money for several years until there's something worth upgrading to.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#27Hopefully we will have more information about these companies when Anthropic IPO filing become public. There's too much speculation without them.
Not if they commit the kinds of market fraud that Musk's companies have.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#28Hang on to your wallet, you ain't seen nothing yet. The true cost of AI won't be revealed until after a large portion of the customer base has become "hooked" on it.
Can you share some use cases of what you think will get companies to become hooked? I'm really failing to see a dependency link emerging either now or in the future. That combined with the relatively easy switching costs. It doesn't bode well for AI companies seeking to create a walled garden.
Teachers in public schools are being told to use AI in their curriculum. In some cases, this means students are being taught not to think (regardless of the intent of the lesson). When prices make this curriculum untenable but the kids already depend on it, that rugpull is going to severely harm a generation of kids whose education was already disturbed in 2020.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#29There seem to two doubtful assumptions being made here:
1. That the API pricing is required to make a profit, rather than being effective market segmentation to make a larger profit.
2. That if subscriptions are loss making, it is not worth having loss leaders.
Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them
#30As usual, there’s no factual basis for the claims other than “I made it up” and author doesn’t seem to have technical experience with ML experience. A lot of weasel words doing all the heavy lifting here.