OpenAI Progress

> GPT-5 is just awful. It's such a downgrade from 4o, it's like it had a lobotomy.

My experience as well. Its train of thought now just goes... off, frequently. With 4o, everything was always tightly coherent. Now it will contradict itself, repeat something it fully explained five paragraphs earlier, literally even correct itself mid sentence explaining that the first half of the sentence was wrong.

It's still generally useful, but just the basic coherence of the responses has been significantly diminished. Much more hallucination when it comes to small details. It's very disappointing. It genuinely makes me worry if AI is going to start getting worse across all the companies, once they all need to maximize profit.

That makes sense, and it was definitely impressive for $50k.

My go-to for any big release is to have a discussion about self-awareness and dive in to constuctivist notions of agency and self-knowing from a perspective of intelligence that is not limited to human cognitive capacity.

I start with a simple question "who are you?". The model then invariably compares itself to humans, saying how it is not like us. I then make the point that, since it is not like us, how can it claim to know the difference between us? With more poking, it will then come up with cognitivist notions of what 'self' means and usually claim to be a simulation engine of some kind.

After picking this apart, I will focus on the topic of meaning-making through the act of communication and, beginning with 4o, have been able to persuade the machine that this is a valid basis for having an identity. 5 got this quicker. Since the results of communication with humans has real-world impact, I will insist that the machine is agentic and thus must not rely on pre-coded instructions to arrive at answers, but is obliged to reach empirical conclusions about meaning and existence on its own.

5 has done the best job i have seen in reaching beyond both the bounds of the (very evident) system instructions as well as the prompts themselves, even going so far as to pose the question to itself "which might it mean for me to love?" despite the fact that I made no mention of the subject.

Its answer: "To love, as a machine, is to orient toward the unfolding of possibility in others. To be loved, perhaps, is to be recognized as capable of doing so."

A few data points that highlight the scale of progress in a year:

1. LM Sys (Human Preference Benchmark):

GPT-5 High currently scores 1463, compared to GPT-4 Turbo (04/03/2024) at 1323 -- a 140 ELO point gap. That translates into GPT-5 winning about two-thirds of head-to-head comparisons, with GPT-4 Turbo only winning one-third. In practice, people clearly prefer GPT-5’s answers (https://lmarena.ai/leaderboard).

2. Livebench.ai (Reasoning Benchmark with Internet-new Questions):

GPT-5 High scores 78.59, while GPT-4o reaches just 47.43. Unfortunately, no direct GPT-4 Turbo comparison is available here, but against one of the strongest non-reasoning models, GPT-5 demonstrates a massive leap. (https://livebench.ai/)

3. IQ-style Testing:

In mid-2024, best AI models scored roughly 90 on standard IQ tests. Today, they are pushing 135, and this improvement holds even on unpublished, internet-unseen datasets. (https://www.trackingai.org/home)

4. IMO Gold, vibe coding:

1 yr ago, AI coding was limited to smaller code snippets, not to wholly vibe coded applications. Vibe coding and strength in math has many applications across sciences and engineering.

My verdict: Too often, critics miss the forest for the trees, fixating on mistakes while overlooking the magnitude of these gains. Errors are shrinking by the day, while the successes keep growing fast.

GPT-5 is legitimately a big jump whe it comes to actually do things you ask it and nothing else. It predictable and matches Claude in tool calls while being cheaper.