Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5 Price Increase: What It Costs

openrouter.ai

Post reply on HN
Print view View original

41–50 of 80 posts

Re: GPT-5.5 Price Increase: What It Costs

#41

Post by NitpickLawyer »

jere wrote: post #25
aurareturn wrote: post #18

Earlier quoted context omitted.

They likely entered the same compute constraint scenario as Anthropic. IE. They had 100 compute units. Demand is 200 units. They have to do a combination of buying more compute, increasing price, lowering limits, etc.

You mean the company that just doubled their rate limits? https://www.anthropic.com/news/higher-limits-spacex

They only did that after they "found" ~300k H100 equivalent compute. Before signing that deal they were severely compute constrained. Especially visible when EU tz was still active and US east would wake up.

Re: GPT-5.5 Price Increase: What It Costs

#42

Post by languid-photic »

lukewarm707 wrote: post #39
languid-photic wrote: post #19

We track performance vs. the all-in cost of completing real engineering tasks, rather than cost per token. [1] Cost per token is a bit misleading because, as others have noted, different models use tokens in different ways. (Aside - This is also why TPS isn't a great metric). We found that 5.5 is about 1.5-2x more expensive overall. On a "Pareto" basis, we only find 5.5 xhigh worth it. At the lower reasoning levels,…

would be interesting to see some other labs: - deepseek v4 pro - glm 5.1 - kimi k2.6 - qwen 3.6 max - xiaomi 2.5 pro - minimax 2.7 - grok

I agree!

So far we have been native harnessmaxxing, which simplifies things a lot.

The configuration space around open models is much larger. Eg which models, capability heterogeneity, which harness, networking, data egress / privacy, etc.

If anyone is getting very good production code out of open models, I'd love to do a user interview to better understand your setup. Email is in my bio.

Re: GPT-5.5 Price Increase: What It Costs

#43

Post by digdugdirk »

languid-photic wrote: post #34
digdugdirk wrote: post #33

Earlier quoted context omitted.

Interesting! I've been thinking about how to create a similar type of evaluation system for myself. How do you handle tweaks to agentic tasks? Say that a model gets pretty close to what you want, so you just need a quick follow up prompt to the original response?

Yes! It depends on the extent of changes needed. If the changes needed are small, I'll apply the best implementation as a foundation and then just iterate directly. If the changes needed are drastic, it usually signals that there was sth wrong/ambiguous/etc in the spec (or the ensemble was too weak, which is rarely the case). In cases like this, I improve the spec and then rerun. If it's in the middle, I'll usually a…

How does that get integrated into the scoring system? I'm imagining a scenario where a cheaper model may get close, but only needs a small follow up to get the desired result. How would this score in comparison to a larger model that got it right the first time - even if it may have been much more expensive overall?

Re: GPT-5.5 Price Increase: What It Costs

#44

Post by BugsJustFindMe »

languid-photic wrote: post #19

We track performance vs. the all-in cost of completing real engineering tasks, rather than cost per token. [1] Cost per token is a bit misleading because, as others have noted, different models use tokens in different ways. (Aside - This is also why TPS isn't a great metric). We found that 5.5 is about 1.5-2x more expensive overall. On a "Pareto" basis, we only find 5.5 xhigh worth it. At the lower reasoning levels,…

It feels pretty weird that your ratings have:

gpt-5-4-high > gpt-5-4-xhigh

gpt-5-4-high > gpt-5-5-high

gpt-5-4 > gpt-5-5

gpt-5-2-high > gpt-5-2-xhigh

No other ratings I've seen show that.

Re: GPT-5.5 Price Increase: What It Costs

#45

Post by thepasch »

languid-photic wrote: post #42
lukewarm707 wrote: post #39

Earlier quoted context omitted.

would be interesting to see some other labs: - deepseek v4 pro - glm 5.1 - kimi k2.6 - qwen 3.6 max - xiaomi 2.5 pro - minimax 2.7 - grok

I agree! So far we have been native harnessmaxxing, which simplifies things a lot. The configuration space around open models is much larger. Eg which models, capability heterogeneity, which harness, networking, data egress / privacy, etc. If anyone is getting very good production code out of open models, I'd love to do a user interview to better understand your setup. Email is in my bio.

With how much vendor harnesses are now actively steering the agent with their own instructions on top of user prompts, I think it’d be super interesting to see a comparison of one of the already tested models - so Opus 4.7 or GPT-5.5 - across a range of different harnesses that aren’t their native. OpenCode, Pi, Hermes, Kilo Code. The most popular coding-focused harnesses, basically.

Re: GPT-5.5 Price Increase: What It Costs

#46

Post by gertlabs »

We observed slightly smaller outputs over long horizon agentic coding for GPT 5.5, at a significant improvement in overall response scores. For one-shot coding responses, GPT 5.5 was actually more verbose than GPT 5.4, but again, the responses were significantly stronger. The expected cost increases reported by OpenRouter seem reasonably accurate (perhaps a bit optimistic), but in my opinion, highly worth it. GPT 5.5 has a pretty wide lead on the #2 model for understanding complex scenarios.

Rankings at https://gertlabs.com/rankings?mode=agentic_coding. See the efficiency chart at the bottom.

Re: GPT-5.5 Price Increase: What It Costs

#47

Post by GorbachevyChase »

barrell wrote: post #23
iceKirin wrote: post #7

I feel that the recent iterations of LLM haven't provided an intuitive qualitative leap. Have they entered a bottleneck period so quickly?

Azure recently discontinued the gpt-4.1 model. I had to move off of this model, and moving to any gpt-5* model was worse (higher failures & less accuracy), and more expensive. I had to rewrite the entire system from high school level prompts to lower elementary school level prompts using non-gpt models. I would say models entered a bottleneck a long time ago. My personal opinion is now they are overfitting newer mode…

I am wondering if everyone is moving to an IPO and striking these bizarre circular deals because they’ve hit the ceiling on what can be done with more compute until a major architectural innovation happens.

Still amazing, but 5.5 does feel like incremental progress with a massive up charge.

Re: GPT-5.5 Price Increase: What It Costs

#48

Post by GorbachevyChase »

cyanydeez wrote: post #21
aurareturn wrote: post #18

Earlier quoted context omitted.

They likely entered the same compute constraint scenario as Anthropic. IE. They had 100 compute units. Demand is 200 units. They have to do a combination of buying more compute, increasing price, lowering limits, etc.

capitalism convinced you that line goes up unless you dont let it eat all the resources.

Yes, Cuba definitely doesn’t have such wild delusions to the benefit of its residents.

Please stop. Critical theory is easy. Something about “X” sucks. Got it. What is the alternative? It’s the completely unserious philosophy of the peanut gallery.

Re: GPT-5.5 Price Increase: What It Costs

#49

Post by iosjunkie »

XCSme wrote: post #16

~3.5x more expensive to run my benchmarks[0]. [0]: https://aibenchy.com/compare/openai-gpt-5-4-medium/openai-gp...

Sure, but it did better on the test, which matches OpenAI's claim. More bang for more buck.

Interestingly, using your tests as a comparison, 5.5 low beats 5.4 medium at a 82% of the cost.[0]

[0]: https://aibenchy.com/compare/openai-gpt-5-4-medium/openai-gp...

Re: GPT-5.5 Price Increase: What It Costs

#50

Post by languid-photic »

BugsJustFindMe wrote: post #44
languid-photic wrote: post #19

We track performance vs. the all-in cost of completing real engineering tasks, rather than cost per token. [1] Cost per token is a bit misleading because, as others have noted, different models use tokens in different ways. (Aside - This is also why TPS isn't a great metric). We found that 5.5 is about 1.5-2x more expensive overall. On a "Pareto" basis, we only find 5.5 xhigh worth it. At the lower reasoning levels,…

It feels pretty weird that your ratings have: gpt-5-4-high > gpt-5-4-xhigh gpt-5-4-high > gpt-5-5-high gpt-5-4 > gpt-5-5 gpt-5-2-high > gpt-5-2-xhigh No other ratings I've seen show that.

Yes, the signal we are measuring is quite different from most evals.

We are measuring sth much closer to: when multiple agents compete on the same spec, which one produces the patch that holds up best in code review?

Most evals are static / synthetic, and for code, generally stop at tests. Test evals are weak proxies for quality since it's difficult to encode qualities like scope creep/churn, codebase fit, maintainability etc in tests. [1]

Almost every agent in a given run can pass tests at this point, but there is large separation during review.

[1] https://voratiq.com/blog/your-workflow-is-the-eval

Post reply on HN