GPT-5.2 • Hacker News Board

Re: GPT-5.2

#121

Post by granzymes » Thu, Dec 11, 2025, 6:59 PM UTC

minimaxir wrote: post #40

josalhor wrote: post #11

From GPT 5.1 Thinking: ARC AGI v2: 17.6% -> 52.9% SWE Verified: 76.3% -> 80% That's pretty good!

Note that GPT 5.2 newly supports a "xhigh" reasoning level, which could explain the better benchmarks. It'll be noteworthy to see the cost-per-task on ARC AGI v2.

> It'll be noteworthy to see the cost-per-task on ARC AGI v2.

Already live. gpt-5.2-pro scores a new high of 54.2% with a cost/task of $15.72. The previous best was Gemini 3 Pro (54% with a cost/task of $30.57).

The best bang-for-your-buck is the new xhigh on gpt-5.2, which is 52.9% for $1.90, a big improvement on the previous best in this category which was Opus 4.5 (37.6% for $2.40).

https://arcprize.org/leaderboard

Re: GPT-5.2

#122

Post by MagicMoonlight » Thu, Dec 11, 2025, 6:59 PM UTC

They’re definitely just training the models on the benchmarks at this point

Re: GPT-5.2

#123

Post by FergusArgyll » Thu, Dec 11, 2025, 7:01 PM UTC

jasonthorsness wrote: post #98

Does anyone have it yet in ChatGPT? I'm still on 5.1 :(.

> We deploy GPT‑5.2 gradually to keep ChatGPT as smooth and reliable as we can; if you don’t see it at first, please try again later.

Re: GPT-5.2

#124

Post by commandar » Thu, Dec 11, 2025, 7:01 PM UTC

tabletcorry wrote: post #7

Slight increase in model cost, but looks like benefits across the board to match. gpt-5.2 $1.75 $0.175 $14.00 gpt-5.1 $1.25 $0.125 $10.00

In particular, the API pricing for GPT-5.2 Pro has me wondering what on earth the possible market for that model is beyond getting to claim a couple of percent higher benchmark performance in press releases.

>Input:

>$21.00 / 1M tokens

>Output:

>$168.00 / 1M tokens

That's the most "don't use this" pricing I've seen on a model.

https://openai.com/api/pricing/

Re: GPT-5.2

#125

Post by simonw » Thu, Dec 11, 2025, 7:01 PM UTC

Wow, there's a lot going on with this pelican riding a bicycle: https://gist.github.com/simonw/c31d7afc95fe6b40506a9562b5e83...

Re: GPT-5.2

#126

Post by stego-tech » Thu, Dec 11, 2025, 7:01 PM UTC

verdverm wrote: post #18

josalhor wrote: post #11

From GPT 5.1 Thinking: ARC AGI v2: 17.6% -> 52.9% SWE Verified: 76.3% -> 80% That's pretty good!

We're also in benchmark saturation territory. I heard it speculated that Anthropic emphasizes benchmarks less in their publications because internally they don't care about them nearly as much as making a model that works well on the day-to-day

These models still consistently fail the only benchmark that matters: if I give you a task, can you complete it successfully without making shit up?

Thus far they all fail. Code outputs don’t run, or variables aren’t captured correctly, or hallucinations are stated as factual rather than suspect or “I don’t know.”

It’s 2000’s PC gaming all over again (“gotta game the benchmark!”).

Re: GPT-5.2

#127

Post by ComputerGuru » Thu, Dec 11, 2025, 7:02 PM UTC

Wish they would include or leak more info about what this is, exactly. 5.1 was just released, yet they are claiming big improvements (on benchmarks, obviously). Did they purposely not release the best they had to keep some cards to play in case of Gemini 3 success or is this a tweak to use more time/tokens to get better output, or what?

Re: GPT-5.2

#128

Post by sheeshe » Thu, Dec 11, 2025, 7:02 PM UTC

JanSt wrote: post #42

The benchmarks are very impressive. Codex and Opus 4.5 are really good coders already and they keep getting better. No wall yet and I think we might have crossed the threshold of models being as good or better than most engineers already. GDPval will be an interesting benchmark and I'll happily use the new model to test spreadsheet (and other office work) capabilities. If they can going like this just a little bit fu…

Ok so why isn’t there mass lay offs ensuing right now?

Re: GPT-5.2

#129

Post by [deleted] » Thu, Dec 11, 2025, 7:02 PM UTC

llmslave wrote: post #19

tabletcorry wrote: post #7

Slight increase in model cost, but looks like benefits across the board to match. gpt-5.2 $1.75 $0.175 $14.00 gpt-5.1 $1.25 $0.125 $10.00

They probably just beefed up compute run time on the what is the same underlying model

[deleted]

Re: GPT-5.2

#130

Post by airstrike » Thu, Dec 11, 2025, 7:02 PM UTC

I feel like if we're going to regulate anything about AI, we should start by regulating (1) what they get to claim to be a "new model" to the public and (2) what changes they are allowed to make at inference before being forced to name it something different.