Earlier quoted context omitted.
commit hash: deadbeef commit author: GPT-4 commit message: ":rocket_emoji: Set all Azure clusters to autoscale in order to resolve flooded message queue"
commit signed by: Samaritan
Person of Interest reference?
OpenAI has temporarily stopped selling the Plus plan
31–40 of 63 posts
Re: OpenAI has temporarily stopped selling the Plus plan
#32Earlier quoted context omitted.
Most obviously this part must be off by 3-4 orders of magnitude: > Let's say the cost of compute is 1 cent per 100B flops (probably a lowball) That's like 0.5ms worth of compute on A100. VMs with an A100 cost a few dollars per hour at cloud providers, i.e. about a cent per second or 0.001 cents per ms. OpenAI would obviously get a far better deal than somebody renting a single GPU for an hour. Look, we know what Open…
Basic common sense would tell you that starting with an assumption and working backwards to justify that assumption is called succumbing to confirmation bias. I sourced my cost/flop estimate from here: https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-fl... I'll acknowledge that a fleet entirely made up of A100 GPUs running at full capacity 100% of the time with Azure's 3-year commitment pricing (likely very…
Uhh... I'm pretty sure that page is about the price of buying an entire GPU that does N GFLOP/S in perpetuity. It has no relevance to the calculation you did, unless you assume the GPU is used for one second and then thrown in the bin.
> I'll acknowledge that a fleet entirely made up of A100 GPUs running at full capacity 100% of the time with Azure's 3-year commitment pricing (likely very low margin) would make this 5 orders of magnitude cheaper. But the sequential nature of transformers makes it difficult to achieve perfect utilization,
I don't think you've thought this through. There's no realistic level of utilization that's going to make up for 5 orders of magnitude. An unimaginably crappy 1% utilization would still leave you off by 3 orders of magnitude.
This very submission about OpenAI having to suspend selling subscriptions makes it pretty clear they're not having trouble with low utilization...
> If ChatGPT plus is $20/mo and each user uses it ~10 times per day, then MS/OpenAI would break even if my estimate were 2 orders of magnitude too high.
You do understand that this doesn't actually support your initial estimate, right? It just makes it more obvious how ridiculous it was.
Re: OpenAI has temporarily stopped selling the Plus plan
#33GPT3 has 175 billion parameters and Altman said 4 would use way more compute than 3. Let's just look at GPT 3. Each forward pass requires 2N=350B flops per token. The computational overhead of the attention mechanism is negligible but the memory overhead of the attention for all users is not, but we can ignore that for now. Let's assume each query involves ~200 tokens of combined input and output. That's 70T flops pe…
>Let's say the cost of compute is 1 cent per 100B flops (probably a lowball).
I might be totally misunderstanding this, but from the source you posted elsewhere[1], GPU Sale Price per Flops is as a little bit higher than 1*10^-11 dollars per flop, or 10^11 flops per dollar, or 10^9 flops per cent. So 1GFlops per cent. But Flops are Floating point operations per second. So you aren't paying 1cent for 100B flops, you're paying 1 cent for a card capable of doing 100B floating point operations per second. So let's say the card lasts 3 years and runs at 100% utilization (I know, it doesn't we can multiply by some efficiency metric later), there's about 9.5*10^7 seconds in 3 years. So that dollar cost isn't for 100B flops, it's for 100B * 9.5 *10^7 ( *some efficiency number). So it's many orders of magnitude cheaper than you're saying.
In reality, the cost of running these things isn't the cost of buying the GPUs, it's the cost of the power to run the GPUs, and that's a totally different calculation we can do.
[1]: https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-fl...*
Re: OpenAI has temporarily stopped selling the Plus plan
#34Earlier quoted context omitted.
> wager this will materially affect Microsoft's next earnings report How?
They're footing the bill for this whole operation. They provide the compute and storage and they're also so heavily invested in OpenAI after their recent $10B round that they can't offload the costs. See my other comment on this post for a conservative cost estimate for each query.
It won't show up on their financials I reckon, they made equity investment into OpenAI - their P&L won't show on Microsoft's. If they have a sweetheart deal with MSFT, Azure revenues aren't going to change. Even if they built a new DC or bought a boatload of GPUs for OpenAI, it will get mixed into their massive Capex and they probably are using existing infrastructure
Re: OpenAI has temporarily stopped selling the Plus plan
#35Earlier quoted context omitted.
They're footing the bill for this whole operation. They provide the compute and storage and they're also so heavily invested in OpenAI after their recent $10B round that they can't offload the costs. See my other comment on this post for a conservative cost estimate for each query.
It won't show up on their financials I reckon, they made equity investment into OpenAI - their P&L won't show on Microsoft's. If they have a sweetheart deal with MSFT, Azure revenues aren't going to change. Even if they built a new DC or bought a boatload of GPUs for OpenAI, it will get mixed into their massive Capex and they probably are using existing infrastructure
They apparently got a lot of Azure credits and Microsoft isn't making any revenue from running OpenAI's workloads.
Re: OpenAI has temporarily stopped selling the Plus plan
#36Have to assume they literally can't get enough GPUs to respond to demand. Wonder if $20/mo is even profitable for them?
It's interesting to think about it this way. I'm sure serious competitors are just around the corner, at least Bard will catch up, No doubt Apple has something in the works, there's little to zero doubt about that. So do they make a huge capital investment hoping that growth will continue at this trajectory, or do they back off the gas pedal ? I bet there's also some contentious ideas going on internally too. If they…
The cynic in me sees they are making a vendor lock-in play, the API is dirt cheap so much so that every other bespoke chatbot or language model is now obsolete - every chat startup is now switching to OpenAI.
Once everyone has their prompts and embeddings tuned and is serving customers, OpenAI can dial up the price and customers chose to keep paying or redo their prompts/embeddings to switch to Bard or the next competitor. It does seem like business as usual if you relied on GPT-3 now ChatGPT, then whatever iteration comes next, probably won't work the same, so its not a heavy heavy lift. Decent platform risk, I believe OpenAI shutdown Codex so now everyone who had a product based on Codex has to switch to ChatGPT
Re: OpenAI has temporarily stopped selling the Plus plan
#37Earlier quoted context omitted.
Most obviously this part must be off by 3-4 orders of magnitude: > Let's say the cost of compute is 1 cent per 100B flops (probably a lowball) That's like 0.5ms worth of compute on A100. VMs with an A100 cost a few dollars per hour at cloud providers, i.e. about a cent per second or 0.001 cents per ms. OpenAI would obviously get a far better deal than somebody renting a single GPU for an hour. Look, we know what Open…
They are just giving away GPT-3 grade queries. In terms of revenue, that's NaN% gross margin! Thus, -10,000% gross margin is quite the improvement. While I personally don't think that they are spending quite that much on compute, burning investor capital to build a customer base is a time honored tradition. Which means we can't really deduce that because -10,000% is such a big number, that can't possibly be their com…
Was this written by gpt?
Re: OpenAI has temporarily stopped selling the Plus plan
#38GPT3 has 175 billion parameters and Altman said 4 would use way more compute than 3. Let's just look at GPT 3. Each forward pass requires 2N=350B flops per token. The computational overhead of the attention mechanism is negligible but the memory overhead of the attention for all users is not, but we can ignore that for now. Let's assume each query involves ~200 tokens of combined input and output. That's 70T flops pe…
This is so confidently wrong it looks like written by GPT itself
Re: OpenAI has temporarily stopped selling the Plus plan
#39Earlier quoted context omitted.
It's interesting to think about it this way. I'm sure serious competitors are just around the corner, at least Bard will catch up, No doubt Apple has something in the works, there's little to zero doubt about that. So do they make a huge capital investment hoping that growth will continue at this trajectory, or do they back off the gas pedal ? I bet there's also some contentious ideas going on internally too. If they…
The cynic in me sees they are making a vendor lock-in play, the API is dirt cheap so much so that every other bespoke chatbot or language model is now obsolete - every chat startup is now switching to OpenAI. Once everyone has their prompts and embeddings tuned and is serving customers, OpenAI can dial up the price and customers chose to keep paying or redo their prompts/embeddings to switch to Bard or the next compe…
I don’t know a lot about codex, but was this a more standard interface used by other providers ?
Re: OpenAI has temporarily stopped selling the Plus plan
#40Earlier quoted context omitted.
It won't show up on their financials I reckon, they made equity investment into OpenAI - their P&L won't show on Microsoft's. If they have a sweetheart deal with MSFT, Azure revenues aren't going to change. Even if they built a new DC or bought a boatload of GPUs for OpenAI, it will get mixed into their massive Capex and they probably are using existing infrastructure
They apparently got a lot of Azure credits and Microsoft isn't making any revenue from running OpenAI's workloads.
Creating credits out of thin air is impossible as all financial transactions must be accounted for. In the case of Azure credits, they will appear as increased revenue on Microsoft's balance sheet. However, if Microsoft decides to price Azure services below cost as part of their agreement with OpenAI, their profit margins will be affected. I highly doubt that is the case because the alleviation of cost is in the form of credits.
I would wager against comment 0 that said that this will materially affect earnings.