Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.2

openai.com

Post reply on HN
Print view View original

901–910 of 1001 posts

Re: GPT-5.2

#901

Post by paulddraper »

EastLondonCoder wrote: post #856
biofox wrote: post #830

Earlier quoted context omitted.

I ask for confidence scores in my custom instructions / prompts, and LLMs do surprisingly well at estimating their own knowledge most of the time.

I’m with the people pushing back on the “confidence scores” framing, but I think the deeper issue is that we’re still stuck in the wrong mental model. It’s tempting to think of a language model as a shallow search engine that happens to output text, but that metaphor doesn’t actually match what’s happening under the hood. A model doesn’t “know” facts or measure uncertainty in a Bayesian sense. All it really does is t…

You have a subtle slight of hand.

You use the word “plausible” instead of “correct.”

Re: GPT-5.2

#902

Post by EastLondonCoder »

ostacke wrote: post #865
EastLondonCoder wrote: post #859

I’ve been using GPT-4o and now 5.2 pretty much daily, mostly for creative and technical work. What helped me get more out of it was to stop thinking of it as a chatbot or knowledge engine, and instead try to model how it actually works on a structural level. The closest parallel I’ve found is Peter Gärdenfors’ work on conceptual spaces, where meaning isn’t symbolic but geometric. Fedorenko’s research on predictive se…

Interesting concept with conceptual spaces, but how does that affect how you work with LLM:s in practice?

I think of it like improvising with a very skilled but slightly alien musician.

If you just hand it a chord chart, it’ll follow the structure. But if you understand the kinds of patterns it tends to favour, the statistical shapes it moves through, you can start composing with it, not just prompting it.

That’s where Gärdenfors helped me reframe things. The model isn’t retrieving facts. It’s traversing a conceptual space. Once you stop expecting grounded truth and start tracking coherence, internal consistency, narrative stability, you get a much better sense of where it’s likely to go off course.

It reminds me of salespeople who speak fluently without being aligned with the underlying subject. Everything sounds plausible, but something’s off. LLMs do that too. You can learn to spot the mismatch, but it takes practice, a bit like learning to jam. You stop reading notes and start listening for shape.

Re: GPT-5.2

#903

Post by hcurtiss »

OrangeMusic wrote: post #757
hcurtiss wrote: post #481

Earlier quoted context omitted.

Neither of these have happened in my use. Those were both the product of some pretty aggressive prompting, and were remedied months ago.

Yet, using this model in any way whatsoever after these episodes seems absolutely crazy to me.

All models have had similar instances. I particularly enjoyed Gemini’s black founders era. The “safety” teams have bent the politics of these tools in ways I don’t trust. Grok does too, but in my experience less so. This has real impacts.

Re: GPT-5.2

#904

Post by krzyk »

kyletns wrote: post #781
phorkyas82 wrote: post #768

Earlier quoted context omitted.

Isn't that what no LLM can provide: being free of hallucinations?

For the record, brains are also not free of hallucinations.

Hallucinations are not bad. It adds some kind of creativity, which is good for e.g. image generation, coding, or story telling.

It is bad only in case of reporting on facts.

Re: GPT-5.2

#905

Post by EastLondonCoder »

paulddraper wrote: post #901
EastLondonCoder wrote: post #856

Earlier quoted context omitted.

I’m with the people pushing back on the “confidence scores” framing, but I think the deeper issue is that we’re still stuck in the wrong mental model. It’s tempting to think of a language model as a shallow search engine that happens to output text, but that metaphor doesn’t actually match what’s happening under the hood. A model doesn’t “know” facts or measure uncertainty in a Bayesian sense. All it really does is t…

You have a subtle slight of hand. You use the word “plausible” instead of “correct.”

That’s deliberate. “Correct” implies anchoring to a truth function the model doesn’t have. “Plausible” is what it’s actually optimising for, and the disconnect between the two is where most of the surprises (and pitfalls) show up.

As someone else put it well: what an LLM does is confabulate stories. Some of them just happen to be true.

Re: GPT-5.2

#906

Post by basisword »

stacktrace wrote: post #785
svara wrote: post #765

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine. The thing that would now make the biggest difference isn't "more intelligence", whatever that might mean, but better grounding. It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations…

> It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations for things, and verifying their claims ends up taking time. And if it's a topic you don't care about enough, you might just end up misinformed. Exactly! One important thing LLMs have made me realise deeply is "No information" is better than false information. The way LLMs pull out completely incorrect explana…

I think the thing even worse than false information is the almost-correct information. You do a quick Google to confirm it's on the right page but find there's an important misunderstanding. These are so much harder to spot I think than the blatantly false.

Re: GPT-5.2

#907

Post by mbesto »

malfist wrote: post #469
AstroBen wrote: post #408

Earlier quoted context omitted.

No-one should have the expectation LLMs are giving correct answers 100% of the time. It's inherent to the tech for them to be confidently wrong Code needs to be checked References need to be checked Any facts or claims need to be checked

According to the benchmarks here they're claiming up to 97% accuracy. That ought to be good enough to trust them right? Or maybe these benchmarks are all wrong

> Or maybe these benchmarks are all wrong

You must be new to LLM benchmarks.

Re: GPT-5.2

#908

Post by tim333 »

elAhmo wrote: post #775

This feels like "could've been an email" type of thing, a very incremental update that just adds one more version. I bet there is literally no one in the world who wanted *one more version of GPT* in the list of available models from OpenAI. "All models" section on https://platform.openai.com/docs/models is quite ridiculous.

It's significant because it looked like they were falling behind Gemini and maybe others.

Re: GPT-5.2

#909

Post by giancarlostoro »

svara wrote: post #765

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine. The thing that would now make the biggest difference isn't "more intelligence", whatever that might mean, but better grounding. It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations…

Yeah in my case I want the coding models to be less stupid, I asked for multiple file uploading, it kept the original button and it added a second one for additional files, when I pointed that out “You're absolutely correct!” Well why didnt you think of it before you cranked out code, I see coding agents as really capable Junior devs its really funny. I dont mind it though, saved me hours on my side project if not weeks worth of work.

Re: GPT-5.2

#910

Post by RHSman2 »

stacktrace wrote: post #785
svara wrote: post #765

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine. The thing that would now make the biggest difference isn't "more intelligence", whatever that might mean, but better grounding. It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations…

> It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations for things, and verifying their claims ends up taking time. And if it's a topic you don't care about enough, you might just end up misinformed. Exactly! One important thing LLMs have made me realise deeply is "No information" is better than false information. The way LLMs pull out completely incorrect explana…

The problem is not the intelligence of the LLM. It is the intelligence and desire to make things easy of the intelligence using them.
Post reply on HN