Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.2

openai.com

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: GPT-5.2

#221

Post by npinsker »

CamperBob2 wrote: post #193
HDThoreaun wrote: post #110

Earlier quoted context omitted.

Arc-AGI is just an iq test. I don’t see the problem with training it to be good at iq tests because that’s a skill that translates well.

Exactly. In principle, at least, the only way to overfit to Arc-AGI is to actually be that smart. Edit: if you disagree, try actually TAKING the Arc-AGI 2 test, then post.

Completely false. This is like saying being good at chess is equivalent to being smart.

Look no farther than the hodgepodge of independent teams running cheaper models (and no doubt thousands of their own puzzles, many of which surely overlap with the private set) that somehow keep up with SotA, to see how impactful proper practice can be.

The benchmark isn’t particularly strong against gaming, especially with private data.

Re: GPT-5.2

#222

Post by zone411 »

I've benchmarked it on the Extended NYT Connections benchmark (https://github.com/lechmazur/nyt-connections/):

The high-reasoning version of GPT-5.2 improves on GPT-5.1: 69.9 → 77.9.

The medium-reasoning version also improves: 62.7 → 72.1.

The no-reasoning version also improves: 22.1 → 27.5.

Gemini 3 Pro and Grok 4.1 Fast Reasoning still score higher.

Re: GPT-5.2

#223

Post by endorphine »

preetamjinka wrote: post #72

It's actually more expensive than GPT-5.1. I've gotten used to prices going down with each latest model, but this time it's gone up. https://platform.openai.com/docs/pricing

Reading this comment, it just occurred to me that we're still in the first phase of the enshittification process.

Re: GPT-5.2

#224

Post by [deleted] »

SparkyMcUnicorn wrote: post #96
elgatolopez wrote: post #29

Earlier quoted context omitted.

Where did you get that from? Cutoff date says august 2025. Looks like a newly pretrained model

If the pretraining rumors are true, they're probably using continued pretraining on the older weights. Right?

[deleted]

Re: GPT-5.2

#225

Post by kingstnap »

preetamjinka wrote: post #72

It's actually more expensive than GPT-5.1. I've gotten used to prices going down with each latest model, but this time it's gone up. https://platform.openai.com/docs/pricing

Flagship models have rarely being cheaper, and especially not on release day. Only a few cases of this really.

Notable exceptions are Deepseek 3.2 and Opus 4.5 and GPT 3.5 Turbo.

The price drops usually are the form of flash and mini models being really cheap and fast. Like when we got o4 mini or 2.0 flash which was a particularly significant one.

Re: GPT-5.2

#226

Post by stego-tech »

verdverm wrote: post #210
stego-tech wrote: post #126

Earlier quoted context omitted.

These models still consistently fail the only benchmark that matters: if I give you a task, can you complete it successfully without making shit up? Thus far they all fail. Code outputs don’t run, or variables aren’t captured correctly, or hallucinations are stated as factual rather than suspect or “I don’t know.” It’s 2000’s PC gaming all over again (“gotta game the benchmark!”).

I'm not sure, here's my anecdotal counter example, was able to get gemini-2.5-flash, in two turns, to understand and implement something I had done separately first, and it found another bug (also that I had fixed, but forgot was in this path) That I was able to have a flash model replicate the same solution I had, to two problems in two turns, it's just the opposite experience of your consistency argument. I'm using…

And therein lies the rub for why I still approach this technology with caution, rather than charge in full steam ahead: variable outputs based on immensely variable inputs.

I read stories like yours all the time, and it encourages me to keep trying LLMs from almost all the major vendors (Google being a noteworthy exception while I try and get off their platform). I want to see the magic others see, but when my IT-brain starts digging in the guts of these things, I’m always disappointed at how unstructured and random they ultimately are.

Getting back to the benchmark angle though, we’re firmly in the era of benchmark gaming - hence my quip about these things failing “the only benchmark that matters.” I meant for that to be interpreted along the lines of, “trust your own results rather than a spreadsheet matrix of other published benchmarks”, but I clearly missed the mark in making that clear. That’s on me.

Re: GPT-5.2

#227

Post by speedgoose »

Trying it now in Vscode Insiders with Github Copilot (codex crashes with HTTP 400 server errors), and it eventually started using sed and grep in shells instead of using the better tools it has access to. I guess this is not an issue to perform well in benchmarks.

Re: GPT-5.2

#228

Post by IAmNotACellist »

enraged_camel wrote: post #181
yousif_123123 wrote: post #152

Why doesn't OpenAI include comparisons to other models anymore?

Because their main competition (Google and Anthropic) have caught up and even started to surpass them, and comparisons would simply drive it home.

Why do they care so much? They're a non-profit dedicated to the betterment of humanity via open access to AI. They have nothing to hide. They have no motivation to lie, or lie by omission.

Re: GPT-5.2

#229

Post by dmos62 »

tempaccount420 wrote: post #108
jstummbillig wrote: post #90

Earlier quoted context omitted.

> For coding tasks, GPT-5.1-Codex-Max is a faster, more capable, and more token-efficient coding variant Hm, yeah, strange. You would not be able to tell, looking at every chart on the page. Obviously not a gotcha, they put it on the page themselves after all, but how does that make sense with those benchmarks?

Coding requires a mindset shift that the -codex fine-tunes provide. Codex will do all kinds of weird stuff like poking in your ~/.cargo ~/go etc. to find docs and trying out code in isolation, these things definitely improve capability.

The biggest advantage of codex variants, for me, is terseness and reduced sicophany. That, and presumably better adherence to requested output formats.

Re: GPT-5.2

#230

Post by sundarurfriend »

zug_zug wrote: post #31

For me the last remaining killer feature of ChatGPT is the quality of the voice chat. Do any of the competitors have something like that?

Are you saying ChatGPT's voice chat is of good quality? Because for me it's one of its most frustrating weaknesses. I vastly prefer voice input to typing, and would love it if the voice chat mode actually worked well.

But apart from the voices being pretty meh, it's also really bad at detecting and filtering out noise, taking vehicle sounds as breaks to start talking in (even if I'm talking much louder at the same time) or as some random YouTube subtitles (car motor = "Thanks for watching, subscribe!").

The speech-to-text is really unreliable (the single-chat Dictate feature gets about 98% of my words correct, this Voice mode is closer to 75%), and they clearly use an inferior model for the AI backend for this too: with the same question asked in this back-and-forth Voice mode and a normal text chat, the answer quality difference is quite stark: the Voice mode answer is most often close to useless. It seems like they've overoptimized it for speed at the cost of quality, to the extent that it feels like it's a year behind in answer reliability and usefulness.

To your question about competitors, I've recently noticed that Grok seems to be much better at both the speech-to-text part and the noise handling, and the voices are less uncanny-valley sounding too. I'd say they also don't have that stark a difference between text answers and voice mode answers, and that would be true but unfortunately mainly because its text answers are also not great with hallucinations or following instructions.

So Grok has the voice part figured out, ChatGPT has the backend AI reliability figured out, but neither provide a real usable voice mode right now.

Post reply on HN