11–20 of 63 posts
Have to assume they literally can't get enough GPUs to respond to demand. Wonder if $20/mo is even profitable for them?
Have to assume they literally can't get enough GPUs to respond to demand. Wonder if $20/mo is even profitable for them?
There's no way it's profitable. I'd wager this will materially affect Microsoft's next earnings report. Hard to tell how open ai itself is affected financially by the explosive growth but a rude awakening is coming for many users and shareholders alike
How?
Earlier quoted context omitted.
GPT-4 is the greatest software developer in the world. I'm sure that it can scale itself easily.
commit hash: deadbeef commit author: GPT-4 commit message: ":rocket_emoji: Set all Azure clusters to autoscale in order to resolve flooded message queue"
GPT3 has 175 billion parameters and Altman said 4 would use way more compute than 3. Let's just look at GPT 3. Each forward pass requires 2N=350B flops per token. The computational overhead of the attention mechanism is negligible but the memory overhead of the attention for all users is not, but we can ignore that for now. Let's assume each query involves ~200 tokens of combined input and output. That's 70T flops pe…
I mean, that might be the cost for a onesie-twosie in Azure, but if you're microsoft and you own the hardware, the cost may be one or more orders of magnitude less than that. (Of course, there's just a limit on the number of GPUs that exist and that Nvidia can pump out.)
Earlier quoted context omitted.
There's no way it's profitable. I'd wager this will materially affect Microsoft's next earnings report. Hard to tell how open ai itself is affected financially by the explosive growth but a rude awakening is coming for many users and shareholders alike
> wager this will materially affect Microsoft's next earnings report How?
The comments are so ignorant it's unreal. I think the OpenAI team is doing their best. I've heard it's actually hard to scale an app to 100+ million users
GPT3 has 175 billion parameters and Altman said 4 would use way more compute than 3. Let's just look at GPT 3. Each forward pass requires 2N=350B flops per token. The computational overhead of the attention mechanism is negligible but the memory overhead of the attention for all users is not, but we can ignore that for now. Let's assume each query involves ~200 tokens of combined input and output. That's 70T flops pe…
GPT3 has 175 billion parameters and Altman said 4 would use way more compute than 3. Let's just look at GPT 3. Each forward pass requires 2N=350B flops per token. The computational overhead of the attention mechanism is negligible but the memory overhead of the attention for all users is not, but we can ignore that for now. Let's assume each query involves ~200 tokens of combined input and output. That's 70T flops pe…
A $7 per query estimate is beyond absurd.
Earlier quoted context omitted.
A $7 per query estimate is beyond absurd.
Which of my estimates do you find absurd? How would you calculate cost?
> Let's say the cost of compute is 1 cent per 100B flops (probably a lowball)
That's like 0.5ms worth of compute on A100. VMs with an A100 cost a few dollars per hour at cloud providers, i.e. about a cent per second or 0.001 cents per ms. OpenAI would obviously get a far better deal than somebody renting a single GPU for an hour.
Look, we know what OpenAI is charging for their commercial service. We don't know how it maps to their actual serving costs; maybe they're breaking even, making a small profit, or making a small loss. But just basic common sense should tell you that they are not operating at -10000% gross margins.
I suspect that just working backwards from the prices to a reasonable range of gross margins is going to produce much better estimates than your methodology would even after the inputs are tuned to be more realistic.
Have to assume they literally can't get enough GPUs to respond to demand. Wonder if $20/mo is even profitable for them?
There's no way it's profitable. I'd wager this will materially affect Microsoft's next earnings report. Hard to tell how open ai itself is affected financially by the explosive growth but a rude awakening is coming for many users and shareholders alike