131–140 of 495 posts
From https://www.theverge.com/command-line-newsletter/759897/sam-... , Sam Altman said: > “If we didn’t pay for training, we’d be a very profitable company.”
These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813
This whole article is built off using DeepSeek R1, which is a huge premise that I don't think is correct. DeepSeek is much more efficient and I don't think it's a valid way to estimate what OpenAI and Anthropic's costs are. https://www.wheresyoured.at/deep-impact/ Basically, DeepSeek is _very_ efficient at inference, and that was the whole reason why it shook the industry when it was released.
Given Gemini efficiency with long context I would bet their attention is very efficient too.
GPT OSS uses fp4, which DeepSeek doesn’t use yet btw.
So no, big labs aren’t behind DeepSeek in efficiency. Not by much at least.
From https://www.theverge.com/command-line-newsletter/759897/sam-... , Sam Altman said: > “If we didn’t pay for training, we’d be a very profitable company.”
Or if they had to pay copyright costs. So much pirated data being repackaged and sold.
This whole article is built off using DeepSeek R1, which is a huge premise that I don't think is correct. DeepSeek is much more efficient and I don't think it's a valid way to estimate what OpenAI and Anthropic's costs are. https://www.wheresyoured.at/deep-impact/ Basically, DeepSeek is _very_ efficient at inference, and that was the whole reason why it shook the industry when it was released.
Earlier quoted context omitted.
Yes. I was really surprised at this myself (author here). If you have some better numbers I'm all ears. Even on my lowly 9070XT I get 20x the tok/s input vs output, and I'm not doing batching or anything locally. I think the cache hit vs miss stuff makes sense at >100k tokens where you start getting compute bound.
I linked to the writeup by Deepseek with their actual numbers from production, and you want "better numbers" than that?! > Each H800 node delivers an average throughput of ~73.7k tokens/s input (including cache hits) during prefilling or ~14.8k tokens/s output during decoding. That's a 5x difference, not 1000x. It also lines up with their pricing, as one would expect. (The decode throughputs they give are roughly equ…
These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813
"Here's the key insight: each forward pass processes ALL tokens in ALL sequences simultaneously." This sounds incorrect, you only process all tokens once, and later incrementally. It's an auto-regressive model after all.
Not during prefill, i.e. the very first token generated in a new conversation. During this forward pass, all tokens in the context are all processed at the same time, and then attention's KV are cached, you still generate a single token, but you need to compute attention from all tokens to all tokens. From that point on every subsequent tokens is processed sequentially in autoregressive way, but because we have the K…
These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813
That doesn't seem compatible with what he stated more recently: > We're profitable on inference. If we didn't pay for training, we'd be a very profitable company. Source: https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... His possible incentives and the fact OpenAI isn't a public company simply make it hard for us to gauge which of these statements is closer to the truth.