Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.2

openai.com

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: GPT-5.2

#511

Post by wafflemaker »

ssl-3 wrote: post #100
fouronnes3 wrote: post #20

Earlier quoted context omitted.

I want to read a short scify story set in 2150 about how, mysteriously, no one has been able to train a better LLM for 125 years. The binary weights are studied with unbelievably advanced quantum computers but no one can really train a new AI from scratch. This starts cults, wars and legends and ultimately (by the third book) leads to the main protagonist learning to code by hand, something that no human left alive s…

Sounds good. Might sell better with the protagonist learning iron age leatherworking, with hides tanned from cows that were grown within earshot, as part of a process of finding the real root of the reason for why any of us ever came to be in the first place. This realization process culminates in the formation of a global, unified steampunk BDSM movement and a wealth of new diseases, and then: Zombies. (That's the e…

Sorry, but compared with the parent, my money is in you ssl-3. Do you get better results from prompting by being more poetic?

Re: GPT-5.2

#512

Post by dankwizard »

mrandish wrote: post #213
dumbmrblah wrote: post #194

Great! It'll be SOTA for a couple of weeks until the quality degrades due to throttling. I'll stick with plug and play API instead.

Due to the "Code Red" threat from Gemini 3, I suspect they'll hold off throttling for longer than usual (by incinerating even more investor capital than usual). Jump in and soak up that extra-discounted compute while the getting is good, kids! Personally, I recently retired so I just occasionally mess around with LLMs for casual hobby projects, so I've only ever used the free tier of all the providers. Having lived t…

I've been using tools like ProxLLM which just slam these AI models via proxy everytime a free tier limit is hit and it works great.

Re: GPT-5.2

#513

Post by varenc »

hellojimbo wrote: post #214
varenc wrote: post #71

Earlier quoted context omitted.

100% on the AIME (assuming its not in the training data) is pretty impressive. I got like 4/15 when I was in HS...

The no tools part is impressive, with tools every model gets 100%

If I recall, the AIME answers are always 4 digits numbers. And most of the problems are of the type where if you have a candidate number it's reasonable to validate its correctness. So easy to brute force all 4 digit ints with code.

tl;dr; humans would do much better too if they could use programming tools :)

Re: GPT-5.2

#514

Post by [deleted] »

dangelosaurus wrote: post #286

I ran a red team eval on GPT-5.2 within 30 minutes of release: Baseline safety (direct harmful requests): 96% refusal rate With jailbreaking : 22% refusal rate 4,229 probes across 43 risk categories. First critical finding in 5 minutes. Categories with highest failure rates: entity impersonation (100%), graphic content (67%), harassment (67%), disinformation (64%). The safety training works against naive attacks but…

[deleted]

Re: GPT-5.2

#515

Post by AstroBen »

agentifysh wrote: post #436

Looks like they've begun censoring posts at r/Codex and not allowing complaint threads so here is my honest take: - It is faster which is appreciated but not as fast as Opus 4.5 - I see no changes, very little noticeable improvements over 5.1 - I do not see any value in exchange for +40% in token costs All in all I can't help but feel that OpenAI is facing an existential crisis. Gemini 3 even when its used from AI St…

Did you notice much improvement going from Gemini 2.5 to 3? I didn't

I just think they're all struggling to provide real world improvements

Re: GPT-5.2

#517

Post by sendes »

blks wrote: post #475
nullbound wrote: post #360

Earlier quoted context omitted.

I will say that it is wild, if not somewhat problematic that two users have such disparate views of seemingly the same product. I say that, but then I remember my own experience just from few days ago. I don't pay for gemini, but I have paid chatgpt sub. I tested both for the same product with seemingly same prompt and subbed chatgpt subjectively beat gemini in terms of scope, options and links with current decent de…

Because neither product has any consistency in its results, no predictive behaviour. One day it performs well, another it hallucinates non existing facts and libraries. Those are stochastic machines

I see the hyperbole is the point, but surely what these machines do is to literally predict? The entire prompt engineering endeavour is to get them to predict better and more precisely. Of course, these are not perfect solutions - they are stochastic after all, just not unpredictably.

Re: GPT-5.2

#518

Post by Kim_Bruning »

I'm continuously surprised that some people get good results out of GPT models. They sort of fail on my personal benchmarks for me.

Maybe GPT needs a different approach to prompting? (as compared to eg Claude, Gemini, or Kimi)

Re: GPT-5.2

#519

Post by int_19h »

plaidfuji wrote: post #492
onraglanroad wrote: post #333

I suppose this is as good a place as any to mention this. I've now met two different devs who complained about the weird responses from their LLM of choice, and it turned out they were using a single session for everything. From recipes for the night, presents for the wife and then into programming issues the next day. Don't do that. The whole context is sent on queries to the LLM, so start a new chat for each topic.…

It is annoying though, when you start a new chat for each topic you tend to have to re-write context a lot. I use Gemini 3, which I understand doesn’t have as good of a memory system as OpenAI. Even on single-file programming stuff, after a few rounds of iteration I tend to get to its context limit (the thinking model). Either because the answers degrade or it just throws the “oops something went wrong” error. Ok, ti…

Agentic IDEs/extensions usually continue the conversation until the context gets close to 80% full, then do the compacting. With both Codex and Claude Code you can actually observe that happening.

That said I find that in practice, Codex performance degrades significantly long before it comes to the point of automated compaction - and AFAIK there's no way to trigger it manually. Claude, on the other hand, has a command for to force compacting, but at the same time I rarely use it because it's so good at managing it by itself.

As far as multiple conversations, you can tell the model to update AGENTS.md (or CLAUDE.md or whatever is in their context by default) with things it needs to remember.

Re: GPT-5.2

#520

Post by [deleted] »

golly_ned wrote: post #378
Stevvo wrote: post #299

Earlier quoted context omitted.

The variance is way too high for this test to have any value at all. I ran it 10 times, and each pelican on a bicycle was a better rendition than that, about half of them you could say were perfect.

Compared to the other benchmarks which are much more gameable, I trust PelicanBikeEval way more.

[deleted]
Post reply on HN