From GPT 5.1 Thinking:
ARC AGI v2: 17.6% -> 52.9%
SWE Verified: 76.3% -> 80%
That's pretty good!
GPT-5.2
11–20 of 1001 posts
Re: GPT-5.2
#12This seems like another "better vibes" release. With the number of benchmarks exploding, random luck means you can almost always find a couple showing what you want to show. I didn't see much concrete evidence this was noticeably better than 5.1 (or even 5.0).
Being a point release though I guess that's fair. I suspect there is also some decent optimizations on the backend that make it cheaper and faster for OpenAI to run, and those are the real reasons they want us to use it.
Re: GPT-5.2
#13It baffles me to see these last 2 announcements (GPT 5.1 as well) devoid of any metrics, benchmarks or quantitative analyses. Could it be because they are behind Google/Anthropic and they don't want to admit it?
(edit: I'm sorry I didn't read enough on the topic, my apologies)
Re: GPT-5.2
#14It baffles me to see these last 2 announcements (GPT 5.1 as well) devoid of any metrics, benchmarks or quantitative analyses. Could it be because they are behind Google/Anthropic and they don't want to admit it? (edit: I'm sorry I didn't read enough on the topic, my apologies)
This isn't the announcement, it's the developer docs intro page to the model - https://openai.com/index/introducing-gpt-5-2/. Still doesn't answer cross-comparison, but at least has benchmark metrics they want to show off.
Re: GPT-5.2
#15"Investors are putting pressure, change the version number now!!!"
I'm quite sad about the S-curve hitting us hard in the transformers. For a short period, we had the excitement of "ooh if GPT-3.5 is so good, GPT-4 is going to be amazing! ooh GPT-4 has sparks of AGI!" But now we're back to version inflation for inconsequential gains.
2025 is the year most Big AI released their first real thinking models
Now we can create new samples and evals for more complex tasks to train up the next gen, more planning, decomp, context, agentic oriented
OpenAI has largely fumbled their early lead, exciting stuff is happening elsewhere
Re: GPT-5.2
#16So GDPval is OpenAI's own benchmark. PDF link: https://arxiv.org/pdf/2510.04374
Re: GPT-5.2
#17Everything is still based on 4 4o still right? is a new model training just too expensive? They can consult deepseek team maybe for cost constrained new models.
I thought whenever the knowledge cutoff increased that meant they’d trained a new model, I guess that’s completely wrong?
Re: GPT-5.2
#18From GPT 5.1 Thinking: ARC AGI v2: 17.6% -> 52.9% SWE Verified: 76.3% -> 80% That's pretty good!
We're also in benchmark saturation territory. I heard it speculated that Anthropic emphasizes benchmarks less in their publications because internally they don't care about them nearly as much as making a model that works well on the day-to-day
Re: GPT-5.2
#19Slight increase in model cost, but looks like benefits across the board to match. gpt-5.2 $1.75 $0.175 $14.00 gpt-5.1 $1.25 $0.125 $10.00
They probably just beefed up compute run time on the what is the same underlying model
Re: GPT-5.2
#20Everything is still based on 4 4o still right? is a new model training just too expensive? They can consult deepseek team maybe for cost constrained new models.
Apparently they have not had a successful pre training run in 1.5 years
I want to read a short scify story set in 2150 about how, mysteriously, no one has been able to train a better LLM for 125 years. The binary weights are studied with unbelievably advanced quantum computers but no one can really train a new AI from scratch. This starts cults, wars and legends and ultimately (by the third book) leads to the main protagonist learning to code by hand, something that no human left alive still knows how to do. Could this be the secret to making a new AI from scratch, more than a century later?