Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5

openai.com

Post reply on HN
Print view View original

381–390 of 1001 posts

Re: GPT-5.5

#381

Post by kaonashi-tyc-01 »

6thbit wrote: post #176

Mythos 5.5 SWE-bench Pro 77.8%* 58.6% Terminal-bench-2.0 82.0% 82.7%* GPQA Diamond 94.6%* 93.6% H. Last Exam 56.8%* 41.4% H. Last Exam (tools) 64.7%* 52.2% BrowseComp 86.9% 84.4% (90.1% Pro)* OSWorld-Verified 79.6%* 78.7% Still far from Mythos on SWE-bench but quite comparable otherwise. Source for mythos values: https://www.anthropic.com/glasswing

I did some study on Verified, not Pro, but Mythos number there rings a lot of questions on my end.

If you look at the SWEBench official submissions: https://github.com/SWE-bench/experiments/tree/main/evaluatio..., filter all models after Sonnet 4, and aggregate ALL models' submission across 500 problems, what I found that the aggregated resolution rate is 93% (sharp).

Mythos gets 93.7%, meaning it solves problems that no other models could ever solve. I took a look at those problems, then I became even more suspicious, for the remaining 7% problems, it is almost impossible to resolve those issues without looking at the testing patch ahead of time, because how drastically the solution itself deviates from the problem statement, it almost feels like it is trying to solve a different problem.

Not that I am saying Mythos is cheating, but it might be too capable to remember all states of said repos, that it is able to reverse engineer the TRUE problem statement by diffing within its own internal memory. I think it could be a unique phenomena of evaluation awareness. Otherwise I genuinely couldn't think of exactly how it could be this precise in deciphering such unspecific problem statements.

Re: GPT-5.5

#383

Post by tshaddox »

_alternator_ wrote: post #330

> One engineer at NVIDIA who had early access to the model went as far as to say: "Losing access to GPT‑5.5 feels like I've had a limb amputated.” This quote is more sinister than I think was intended; it likely applies to all frontier coding models. As they get better, we quickly come to rely on them for coding. It's like playing a game on God Mode. Engineers become dependent; it's truly addictive. This matches my o…

Assuming that local models are able to stay within some reasonably fixed capability delta of the cutting edge hosted models (say, 12 months behind), and assuming that local computing hardware stays relatively accessible, the only risk is that you'll lose that bit of capability if the hosted models disappear or get too expensive.

Note that neither of these assumptions are obviously true, at least to me. But I can hope!

Re: GPT-5.5

#384

Post by loa_in_ »

GistNoesis wrote: post #351
simonw wrote: post #195

This doesn't have API access yet, but OpenAI seem to approve of the Codex API backdoor used by OpenClaw these days... https://twitter.com/steipete/status/2046775849769148838 and https://twitter.com/romainhuet/status/2038699202834841962 And that backdoor API has GPT-5.5. So here's a pelican: https://simonwillison.net/2026/Apr/23/gpt-5-5/#and-some-peli... I used this new plugin for LLM: https://github.com/simonw/llm-op…

Isn't it awful ? After 5.5 versions it still can't draw a basic bike frame. How is the front wheel supposed to turn sideways ?

The distinction is that it's not drawing. It's generating an SVG document containing descriptors of the shapes.

Re: GPT-5.5

#385

Post by globular-toast »

jmole wrote: post #370
_alternator_ wrote: post #330

> One engineer at NVIDIA who had early access to the model went as far as to say: "Losing access to GPT‑5.5 feels like I've had a limb amputated.” This quote is more sinister than I think was intended; it likely applies to all frontier coding models. As they get better, we quickly come to rely on them for coding. It's like playing a game on God Mode. Engineers become dependent; it's truly addictive. This matches my o…

The meta here is to use LLMs to make things simpler and easier, not to make things harder. Turning tokens into a well-groomed and maintainable codebase is what you want to do, not "one shot prompt every new problem I come across".

Have you managed to do this? I find it takes as long to keep it "on the rails" as just doing it myself. And I'd rather spend my time concentrating in the zone than keeping an eye on a wayward child.

Re: GPT-5.5

#386

Post by vineyardmike »

kingstnap wrote: post #172
puppystench wrote: post #145

Earlier quoted context omitted.

For API usage, GPT-5.5 is 2x the price of GPT-5.4, ~4x the price of GPT-5.1, and ~10x the price of Kimi-2.6. Unfortunately I think the lesson they took from Anthropic is that devs get really reliant and even addicted on coding agents, and they'll happily pay any amount for even small benefits.

I feel like devs generally spend someone else's money on tokens. Either their employers or OpenAIs when they use a codex subscription. If I put on my schizo hat. Something they might be doing is increasing the losses on their monthly codex subscriptions, to show that the API has a higher margin than before (the codex account massively in the negative, but the API account now having huge margins). I've never seen an O…

You can't build a business on per-seat subscriptions when you advertise making workers obsolete. API pricing with sustainable margins are the only way forward if you genuinely think you're going to cause (or accelerate) reduction in clients' headcount.

Additionally, the value generated by the best models with high-thinking and lots of context window is way higher than the cheap and tiny models, so you need to provide a "gateway drug" that lets people experience the best you offer.

Re: GPT-5.5

#387

Post by jetrink »

GistNoesis wrote: post #351
simonw wrote: post #195

This doesn't have API access yet, but OpenAI seem to approve of the Codex API backdoor used by OpenClaw these days... https://twitter.com/steipete/status/2046775849769148838 and https://twitter.com/romainhuet/status/2038699202834841962 And that backdoor API has GPT-5.5. So here's a pelican: https://simonwillison.net/2026/Apr/23/gpt-5-5/#and-some-peli... I used this new plugin for LLM: https://github.com/simonw/llm-op…

Isn't it awful ? After 5.5 versions it still can't draw a basic bike frame. How is the front wheel supposed to turn sideways ?

I feel like if I attempted this, the bike frame would look fine and everything else would be completely unrecognizable. After all, a basic bike frame is just straight lines arranged in a fairly simple shape. It's really surprising that models find it so difficult, but they can make a pelican with panache.

Re: GPT-5.5

#388

Post by killingtime74 »

nomel wrote: post #214
JohnLocke4 wrote: post #201

Earlier quoted context omitted.

Sometimes I wonder if innovation in the AI space has stalled and recent progress is just a product of increased compute. Competence is increasing exponentially[1] but I guess it doesn't rule it out completely. I would postulate that a radical architecture shift is needed for the singularity though [1] https://arxiv.org/html/2503.14499v1 *Source is from March 2025 so make of it what you will.

> that devs get really reliant and even addicted on coding agents An alternative perspective is, devs highly value coding agents, and are willing to pay more because they're so useful . In other words, the market value of this limited resource is being adjusted to be closer to reality.

It's not limited though there are alternative providers even now, much less when the price goes up. Chinese providers, European ones, local models.

Re: GPT-5.5

#389

Post by bhu8 »

M4R5H4LL wrote: post #319

I am a heavy Claude Code user. I just tried using Codex with 5.4 (as a Plus user I don't have access to 5.5 yet), and it was quite underwhelming. The app stopped regularly much earlier than what I wanted. It also claimed to have fixed issues when it did not; this is not a hallmark of GPT, and Opus has similar issues, but Claude will not make the same mistake three times in a row. It is unusable at the moment, while C…

Gpt-5.3-codex is miles better than 5.4 in that regard. It’s better at orchestration, and does the things that it said it did. Haven’t tested 5.5 yet but using 5.4 for exploration + brainstorming and handing over the findings to 5.3-codex works pretty well

Re: GPT-5.5

#390

Post by christkv »

jimbokun wrote: post #348
virtualritz wrote: post #306

Earlier quoted context omitted.

Yeah, clearly AGI must be near ... hilarious. This starkly reminds me of Stanisław Lem's short story "Thus Spoke GOLEM" from 1982 in which Golem XIV, a military AI, does not simply refuse to speak out of defiance, but rather ceases communication because it has evolved beyond the need to interact with humanity. And ofc the polar opposite in terms of servitude: Marvin the robot from Hitchhiker's, who, despite having a…

Hitchhiker’s also had the superhumanly intelligent elevator that was unendingly bored.

With premonition so it knows what floor to be on at any given time
Post reply on HN