Earlier quoted context omitted.
Gemini is the best model in the world. Gemini is the worst web app in the world. Somehow those two things are coexisting. The web devs in their UI team have really betrayed the hard work of their ML and hardware colleagues. I don't say this lightly - I say this after having paid attention to critical bugs, more than I can count on one hand, that persisted for over a year. They either don't care or are grossly incompe…
Well said. Google is best in pure AI research, both quality and volume. They have sucked at productization for years. Not not just AI but other products as well. Real mystery.
OpenAI dropped the price of o3 by 80%
171–180 of 518 posts
Re: OpenAI dropped the price of o3 by 80%
#172Earlier quoted context omitted.
Is this what happened to Gemini 2.5 Pro? It used to be very good, but it's started struggling on basic tasks. The thing that gets me is it seems to be lying about fetching a web page. It will say things are there that were never on any version of the page and it sometimes takes multiple screenshots of the page to convince it that it's wrong.
The Aider discord community has proposed and disproven the theory that 2.5 Pro became worse, several times, through many benchmark runs. It had a few bugs here or there when they pushed updates, but it didn't get worse.
My question is not whether this is true (it is) but why it's happening.
I am willing to believe the aider community has found that Gemini has maintained approximately equivalent performance on fixed benchmarks. That's reasonable considering they probably use a/b testing on benchmarks to tell them whether training or architectural changes need to be reverted.
But all versions of aider I've tested, including the most recent one, don't handle Gemini correctly so I'm skeptical that they're the state of the art with respect to bench-marking Gemini.
Re: OpenAI dropped the price of o3 by 80%
#173...how? I'd understand a 20-30% price drop from infra improvements for a model as-is, but 80%? I wonder if "we quantized it lol" would classify as false advertising for modern LLMs.
Deepseek made a few major innovations allowing them to achieve major compute efficiency and then published them. My guess is that OpenAI just implemented these themselves.
Re: OpenAI dropped the price of o3 by 80%
#174Note that they have not actually dropped the price yet: https://x.com/OpenAIDevs/status/1932463601119637532 > We’ll post to @openaidevs once the new pricing is in full effect. In $10… 9… 8… There is also speculation that they are only dropping the input price, not the output price (which includes the reasoning tokens).
I think that was a joke. New pricing is already in place: Input: $2.00 / 1M tokens Cached input: $0.50 / 1M tokens Output: $8.00 / 1M tokens https://openai.com/api/pricing/ Now cheaper than gpt-4o and same price as gpt-4.1 (!).
Re: OpenAI dropped the price of o3 by 80%
#175Earlier quoted context omitted.
I think that was a joke. New pricing is already in place: Input: $2.00 / 1M tokens Cached input: $0.50 / 1M tokens Output: $8.00 / 1M tokens https://openai.com/api/pricing/ Now cheaper than gpt-4o and same price as gpt-4.1 (!).
> Now cheaper than gpt-4o and same price as gpt-4.1 (!). This is where the naming choices get confusing. "Should" o3 cost more or less than GPT-4.1? Which is more capable? A generation 3 of tech intuitively feels less advanced than a 4.1 of a (similar) tech.
Re: OpenAI dropped the price of o3 by 80%
#176how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…
Re: OpenAI dropped the price of o3 by 80%
#177 Yesterday: Today
------------- -------------
Price Price
Input: Input:
$10.00 / 1M tokens $2.00 / 1M tokens
Cached input: Cached input:
$2.50 / 1M tokens $0.50 / 1M tokens
Output: Output:
$40.00 / 1M tokens $8.00 / 1M tokens
https://archive.is/20250610154009/https://openai.com/api/pri...Re: OpenAI dropped the price of o3 by 80%
#178Earlier quoted context omitted.
All costs are not equal. There is a classic pattern of dogfights for winner-take-most product categories where the long term winner does the best job of acquiring customers at the expense of things like "engineering to reduce costs". I have no idea how the AI space is going to shake out, but if I had to pick between OpenAI's mindshare in the broadest possible cohort of users vs. best/most efficient model, I'd pick th…
We don't even know yet if the model is the product though, and if OpenAI is the company that will make the AI product/model, (chat that keeps expanding into other functionalities and capabilities) or will it be 10,000 companies using the OpenAI models. (well, it's probably both, but in what proportion of revenue)
Again: I don't know. I've got no predictions. I'm just saying that the logic where OpenAI is outcompeted on models themselves and thus automatically lose does not hold automatically.
Re: OpenAI dropped the price of o3 by 80%
#179Why does OpenAI require me to verify my "organization" (which requires my state issued ID) to use o3?
Re: OpenAI dropped the price of o3 by 80%
#180I don't know if this is OpenAI's intention, but the little message "you've reached your usage limit!" is actively disincentivizing me from subscribing. For my purposes, the free model is more than good enough; the difference before and after is negligible. I honestly wouldn't pay a dollar. That said, I'm absolutely willing to hear people out on "value-adds" I am missing out on; I'm not a knee-jerk hater (For context,…
I've never used anything like it. I think new Claude is similarly capable