o3 is very much needed in VSCode GitHub CoPilot for Ask/Edit/Agent modes. It is sorely missing there.
OpenAI dropped the price of o3 by 80%
161–170 of 518 posts
Re: OpenAI dropped the price of o3 by 80%
#162Has anyone noticed that OpenAI has become "lazy"? When I ask questions now it will not give me a complete file or fix. Instead it tells me what I should do and I need to ask a second or third time to just do the thing I asked. I don't see this happening with for example deepseek. Is it possible they are saving on resources by having it answer that way?
Yeah, our models are sometimes too lazy. It’s not intentional, and future models will be less lazy.
When I worked at Netflix I sometimes heard the same speculation about intentionally bad recommendations, which people theorized would lower streaming and increase profit margins. It made even less sense there as streaming costs are usually less than a penny. In reality, it’s just hard to make perfect products!
(I work at OpenAI.)
Re: OpenAI dropped the price of o3 by 80%
#163Earlier quoted context omitted.
Gemini is simply that good. I’m trying out Claude 4 every now and then and go back to Gemini to fix its mess…
Gemini is the best model in the world. Gemini is the worst web app in the world. Somehow those two things are coexisting. The web devs in their UI team have really betrayed the hard work of their ML and hardware colleagues. I don't say this lightly - I say this after having paid attention to critical bugs, more than I can count on one hand, that persisted for over a year. They either don't care or are grossly incompe…
Well said.
Google is best in pure AI research, both quality and volume. They have sucked at productization for years. Not not just AI but other products as well. Real mystery.
Re: OpenAI dropped the price of o3 by 80%
#164Earlier quoted context omitted.
Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
Updated yesterday... https://aider.chat/docs/leaderboards/
That Deepseek price is always hilarious to see in these charts.
Re: OpenAI dropped the price of o3 by 80%
#165Earlier quoted context omitted.
The problem is your costs also scale with revenue. Ideally you want to have control costs as you scale (the first you build is expensive, but as you make more your costs come down). For OpenAI, the more people use the product, the same you spend on compute unless they can supplement it with another ways of generating revenue. I dont unfortunately think OpenAI will be able to hit sustained profitability (see Netflix f…
All costs are not equal. There is a classic pattern of dogfights for winner-take-most product categories where the long term winner does the best job of acquiring customers at the expense of things like "engineering to reduce costs". I have no idea how the AI space is going to shake out, but if I had to pick between OpenAI's mindshare in the broadest possible cohort of users vs. best/most efficient model, I'd pick th…
We don't even know yet if the model is the product though, and if OpenAI is the company that will make the AI product/model, (chat that keeps expanding into other functionalities and capabilities) or will it be 10,000 companies using the OpenAI models. (well, it's probably both, but in what proportion of revenue)
Re: OpenAI dropped the price of o3 by 80%
#166how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…
I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version. https://x.com/hyperknot/status/1932476190608036243
Is that input tokens or output tokens/s?
Re: OpenAI dropped the price of o3 by 80%
#167Earlier quoted context omitted.
I mean sure, it's very promising if OpenAI's future is your only metric. It gets notably darker if you look at the broader picture of ChatGPT (and company)'s impact on our society. * We have people uploading tons of zero-effort slop pieces to all manner of online storefronts, and making people less likely to buy overall because they assume everything is AI now * We have an uncomfortable community of, to be blunt, act…
Dying for a reference on the cult stuff, a quick search didn’t provide anything interesting.
https://futurism.com/chatgpt-mental-health-crises, which references the more famous https://www.rollingstone.com/culture/culture-features/ai-spi... but is a newer article.
Re: OpenAI dropped the price of o3 by 80%
#168Earlier quoted context omitted.
I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.
I've heard lots of people say that, but no objective reproducible benchmarks confirm such a thing happening often. Could this simply be a case of novelty/excitement for a new model fading away as you learn more about its shortcomings?
I assumed it was because the first week revealed a ton of safety issues that they then "patched" by adjusting the system prompt, and thus using up more inference tokens on things other than the user's request.
Re: OpenAI dropped the price of o3 by 80%
#169Earlier quoted context omitted.
I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.
It seems that least Google is overselling their compute capacity. You pay monthly fee, but Gemini is completely jammed 5-6 hours when North America is working.
When you say "jammed," how do you mean?
Re: OpenAI dropped the price of o3 by 80%
#170it used to take decades of erosion to make google search a hot mess, now that everything's happening in light speed, we get days for AI models to decay to the point of hot mess again..