Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI dropped the price of o3 by 80%

twitter.com

Post reply on HN
Print view View original

101–110 of 518 posts

Re: OpenAI dropped the price of o3 by 80%

#101

Post by carter-0 »

behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

An OpenAI researcher claims it's the exact same model on X: https://x.com/aidan_mclau/status/1932507602216497608

Re: OpenAI dropped the price of o3 by 80%

#102

Post by slashdev »

rvnx wrote: post #27
candiddevmike wrote: post #16

It's going to be a race to the bottom, they have no moat.

Especially now that they are second in the race (behind Anthropic) and lot of free-to-download and free-to-use models are now starting to be viable competitors. Once new MacBooks and iPhones have enough memory onboard this is going to be a disaster for OpenAI and other providers.

Thrid for coding, after Anthropic, and Gemini, which was leading last I checked.

Re: OpenAI dropped the price of o3 by 80%

#103

Post by Kranar »

Tiberium wrote: post #57
lispisok wrote: post #18

Earlier quoted context omitted.

I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.

I've heard lots of people say that, but no objective reproducible benchmarks confirm such a thing happening often. Could this simply be a case of novelty/excitement for a new model fading away as you learn more about its shortcomings?

I used to think the models got worse over time as well but then I checked my chat history and what I noticed isn't that ChatGPT gets worse, it's that my standards and expectations increase over time.

When a new model comes out I test the waters a bit with some more ambitious queries and get impressed when it can handle them reasonably well. Over time I take it for granted and then just expect it to be able to handle ever more complex queries and get dissappointed when I hit a new limit.

Re: OpenAI dropped the price of o3 by 80%

#104

Post by SparkyMcUnicorn »

resters wrote: post #88
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

It's probably optimized in some way, but if the optimizations degrade performance, let's hope it is reflected in various benchmarks. One alternative hypothesis is that it's the same model, but in the early days they make it think "harder" and run a meta-process to collect training data for reinforcement learning for use on future models.

It's a bit dated now, but it would be cool if people submitted PRs for this one: https://aider.chat/docs/leaderboards/by-release-date.html

Re: OpenAI dropped the price of o3 by 80%

#105

Post by Szpadel »

candiddevmike wrote: post #16

It's going to be a race to the bottom, they have no moat.

for sure they are no longer clear winners, but they try to be just barely on top of others.

right now new Gemini surpassed their o3 (barely) in benchmarks for significantly less money so they cut pricing to be still competitive.

I bet they didn't released o4 not because it's not competitive, but because they are doing Nvidia game: release new product that is just enough better to convince people to buy it. so IMO they are holding full o4 model to have something to release after competition release something better that their top horse

Re: OpenAI dropped the price of o3 by 80%

#106

Post by johan914 »

lvl155 wrote: post #42

Google has been catching up. Funny how fast this space is evolving. Just a few months ago, it was all about DeepSeek.

I have been using Google’s models the past couple months, and was surprised to see how sycophantic chatGPT is now. It’s not just at the start or end of responses, it’s interspaced within the markdown, with little substance. Asking it to change its style makes it overuse technical terms.

Re: OpenAI dropped the price of o3 by 80%

#107

Post by arealaccount »

ToucanLoucan wrote: post #77
coffeecoders wrote: post #51

Despite the popular take that LLMs have no moat and are burning cash, I find OpenAI's situation really promising. Just yesterday, they reported an annualized revenue run rate of 10B. Their last funding round in March valued them at 300B. Despite losing 5B last year, they are growing really fast - 30x revenue with over 500M active users. It reminds me a lot of Uber in its earlier years—fast growth, heavy investment, b…

I mean sure, it's very promising if OpenAI's future is your only metric. It gets notably darker if you look at the broader picture of ChatGPT (and company)'s impact on our society. * We have people uploading tons of zero-effort slop pieces to all manner of online storefronts, and making people less likely to buy overall because they assume everything is AI now * We have an uncomfortable community of, to be blunt, act…

Dying for a reference on the cult stuff, a quick search didn’t provide anything interesting.

Re: OpenAI dropped the price of o3 by 80%

#108

Post by SparkyMcUnicorn »

ants_everywhere wrote: post #84
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

Is this what happened to Gemini 2.5 Pro? It used to be very good, but it's started struggling on basic tasks. The thing that gets me is it seems to be lying about fetching a web page. It will say things are there that were never on any version of the page and it sometimes takes multiple screenshots of the page to convince it that it's wrong.

The Aider discord community has proposed and disproven the theory that 2.5 Pro became worse, several times, through many benchmark runs.

It had a few bugs here or there when they pushed updates, but it didn't get worse.

Re: OpenAI dropped the price of o3 by 80%

#109

Post by jacob019 »

snarf21 wrote: post #79
bitpush wrote: post #63

Earlier quoted context omitted.

Many would say Google's Gemini models are SOTA, although Claude seems to be doing well with coding tasks.

Gemini has been better than Claude for me on a coding project. Claude kept telling me it update some code but the update wasn't in the output. Like, I had to re-prompt just for updated output 5 times in a row.

I break out Gemini 2.5 pro when Claude gets stuck, it's just so slow and verbose. Claude follows instructions better and seems to better understand it's role in agentic workflows. Gemini does something different with the context, it has a deeper understanding of the control flow and can uncover edge case bugs that Claude misses. o3 seems better at high level thinking and planning, questioning if it should it be done and whether the challenge actually matches the need. They're kind of like colleagues with unique strengths. o3 does well with a lot of things, I just haven't used it as much because of the cost. Will probably use it more now.

Re: OpenAI dropped the price of o3 by 80%

#110

Post by SparkyMcUnicorn »

esafak wrote: post #12
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

Are there any benchmarks that track historical performance?

Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
Post reply on HN