Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.2

openai.com

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: GPT-5.2

#881

Post by sheeshe »

kznewman wrote: post #871
EastLondonCoder wrote: post #856

Earlier quoted context omitted.

I’m with the people pushing back on the “confidence scores” framing, but I think the deeper issue is that we’re still stuck in the wrong mental model. It’s tempting to think of a language model as a shallow search engine that happens to output text, but that metaphor doesn’t actually match what’s happening under the hood. A model doesn’t “know” facts or measure uncertainty in a Bayesian sense. All it really does is t…

Solid agree. Hallucination for me IS the LLM use case. What I am looking for are ideas that may or may not be true that I have not considered and then I go try to find out which I can use and why.

In essence it is a thing that is actually promoting your own brain… seems counter intuitive but that’s how I believe this technology should be used.

Re: GPT-5.2

#882

Post by neuah »

AstroBen wrote: post #515
agentifysh wrote: post #436

Looks like they've begun censoring posts at r/Codex and not allowing complaint threads so here is my honest take: - It is faster which is appreciated but not as fast as Opus 4.5 - I see no changes, very little noticeable improvements over 5.1 - I do not see any value in exchange for +40% in token costs All in all I can't help but feel that OpenAI is facing an existential crisis. Gemini 3 even when its used from AI St…

Did you notice much improvement going from Gemini 2.5 to 3? I didn't I just think they're all struggling to provide real world improvements

Using it in a specialized subfield of neuroscience, Gemini 3 w/ thinking is a huge leap forward in terms of knowledge and intelligence (with minimal hallucinations). I take it that the majority of people on here are software engineers. If you're evaluating it on writing boilerplate code, you probably have to squint to see differences between the (excellent) raw model performances. whereas in more niche edge cases there is more daylight between them.

Re: GPT-5.2

#883

Post by SecretDreams »

phorkyas82 wrote: post #768
svara wrote: post #765

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine. The thing that would now make the biggest difference isn't "more intelligence", whatever that might mean, but better grounding. It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations…

Isn't that what no LLM can provide: being free of hallucinations?

Find me a human that doesn't occasionally talk out of their ass =[

Re: GPT-5.2

#884

Post by balder1991 »

incrudible wrote: post #825
robocat wrote: post #801

Earlier quoted context omitted.

> wrong or misleading explanations Exactly the same issue occurs with search. Unfortunately not everybody knows to mistrust AI responses, or have the skills to double-check information.

If somebody asks a question on Stackoverflow, it is unlikely that a human who does not know the answer will take time out of their day to completely fabricate a plausible sounding answer.

At least it used to be true.

Re: GPT-5.2

#885

Post by jaxn »

incrudible wrote: post #825
robocat wrote: post #801

Earlier quoted context omitted.

> wrong or misleading explanations Exactly the same issue occurs with search. Unfortunately not everybody knows to mistrust AI responses, or have the skills to double-check information.

If somebody asks a question on Stackoverflow, it is unlikely that a human who does not know the answer will take time out of their day to completely fabricate a plausible sounding answer.

People are confidently incorrect all the time. It is very likely that people will make up plausible sounding answers on StackOverflow.

You and I have both taken time out of our days to write plausible sounding answers that are essentially opposing hallucinations.

Re: GPT-5.2

#887

Post by okthrowman283 »

getnormality wrote: post #725
onraglanroad wrote: post #333

I suppose this is as good a place as any to mention this. I've now met two different devs who complained about the weird responses from their LLM of choice, and it turned out they were using a single session for everything. From recipes for the night, presents for the wife and then into programming issues the next day. Don't do that. The whole context is sent on queries to the LLM, so start a new chat for each topic.…

In my recent explorations [1] I noticed it got really stuck on the first thing I said in the chat, obsessively returning to it as a lens through which every new message had to be interpreted. Starting new sessions was very useful to get a fresh perspective. Like a human, an AI that works on a writing piece with you is too close to the work to see any flaw. [1] https://renormalize.substack.com/p/on-renormalization

Interesting I’ve noticed the same behavior with Gemini 3.0 but not with Claude, and Gemini 2.5 did not have this behavior. I wonder what tuning is optimising for here.

Re: GPT-5.2

#888

Post by balder1991 »

conception wrote: post #877
BatteryMountain wrote: post #860

Earlier quoted context omitted.

My biggest problem with LLM's at this point is that they produce different and inconsistent results or behave differently, given the same prompt. The better grounding would be amazing at this point. I want to give an LLM the same prompt on different days and I want to be able to trust that it will do the same thing as yesterday. Currently they misbehave multiple times a week and I have to manually steer it a bit whic…

You need to change the temperature to 0 and tune your prompts for automated workflows.

It doesn’t really solve it as a slight shift in the prompt can have totally unpredictable results anyway. And if your prompt is always exactly the same, you’d just cache it and bypass the LLM anyway.

What would really be useful is a very similar prompt should always give a very very similar result.

Re: GPT-5.2

#889

Post by coder543 »

nathants wrote: post #710
coder543 wrote: post #565

Earlier quoted context omitted.

https://platform.openai.com/docs/models/gpt-5.2 400k, not 256k.

400 - 128 = 272. Codex cli source.

If you want to be able to generate up to 128k tokens in one go successfully, then yes, that math checks out.

Re: GPT-5.2

#890

Post by fragmede »

kiliankoe wrote: post #849
drclau wrote: post #845

Earlier quoted context omitted.

How do you know the confidence scores are not hallucinated as well?

They are, the model has no inherent knowledge about its confidence levels, it just adds plausible-sounding numbers. Obviously they _can_ be plausible, but trusting these is just another level up from trusting the original output. I read a comment here a few weeks back that LLMs always hallucinate, but we sometimes get lucky when the hallucinations match up with reality. I've been thinking about that a lot lately.

In science, before LLMs, there's this saying: all models are wrong, some are useful. We model, say, gravity as 9.8m/s² on Earth, knowing full well that it doesn't hold true across the universe, and we're able to build things on top of that foundation. Whether that foundation is made of bricks, or is made of sand, for LLMs, is for us to decide.
Post reply on HN