Live data from Hacker News

OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

developers.openai.com

71–80 of 174 posts

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#71
post #37

Just tried it out for a prod issue was experiencing. Claude never does this sort of thing, I had it write an update statement after doing some troubleshooting, and I said “okay let’s write this in a transaction with a rollback” and GPT-5.5 gave me the old “okay, BEGIN TRAN; -- put the query here commit; I feel like I haven’t had to prod a model to actually do what I told it to in awhile so that was a shock. I guess t…

OpenAI is the first company that has reached a level of intelligence so high, the model has finally become smart enough to make YOU do all the work. Emergent behavior in action.

All earnesty aside, OpenAI’s oddly specific singular focus on “intelligence per token” (also in the benchmarks) that literally noone else pushes so hard eerily reminds me of Apple’s Macbook anorexia era pre-M1. One metric to chase at the cost of literally anything else. GPT-5.3+ are some of the smartest models out there and could be a pleasure to work with, if they weren’t lazy bastards to the point of being completely infuriating.

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#72
post #48

Earlier quoted context omitted.

I feel like the last 2-3 generations of models (after gpt-5.3-codex) didn't really improve much, just changed stuff around and making different tradeoffs.

I disagree, it improved enormously especially at staying consistent for long-tasks, I have a task running for 32 days (400M+ tokens) via Codex and that's only since gpt-5.4

...what? what kind of a task are you running?

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#73
post #37

Just tried it out for a prod issue was experiencing. Claude never does this sort of thing, I had it write an update statement after doing some troubleshooting, and I said “okay let’s write this in a transaction with a rollback” and GPT-5.5 gave me the old “okay, BEGIN TRAN; -- put the query here commit; I feel like I haven’t had to prod a model to actually do what I told it to in awhile so that was a shock. I guess t…

GPT-5.5 shatters benchmarks for amount of faith it puts in the user.

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#74
post #56

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...

You even traveled in time to deliver us this benchmark. I really like this benchmarking. Have you evaluated the judge benchmark somehow? I'd love to setup my own similar benchmark.

Haha, just fixed the date!

I haven't evaluated the judge benchmark. You have everything needed in the repo to do so though, so be my guest. It took me a bit of time to put all this together and won't have much more time to dedicate to it before a couple of weeks.

BTW, if you explore the repo, sorry for all the French files...

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#76
post #56

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...

Seems like benchmark for how good a model is for vibe coding.

Your prompt is extremely slim yet you score it on a bunch of features.

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#77
post #56

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...

gemma4-e4b is 50% better than gemma4-26b in your benchmark, something's wrong

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#78
post #56

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...

Seems like benchmark for how good a model is for vibe coding. Your prompt is extremely slim yet you score it on a bunch of features.

Yes, the prompt is slim by design. I might be wrong, but the point was to see what the model can do "on it's own".

The eval prompt is quite extensive: https://github.com/guilamu/llms-wordpress-plugin-benchmark/b...

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#79
post #56

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...

gemma4-e4b is 50% better than gemma4-26b in your benchmark, something's wrong

Yes those two models were tested on my own PC (local inference using my own CPU/GPU). So something my be bugged on my setup. gemma4-26b should be far better than gemma4-e4b.

Re: OpenAI releases GPT-5.5 and GPT-5.5 Pro in the API

#80
post #70
post #61

Earlier quoted context omitted.

Your benchmark has Opus 4.7 performing significantly worse than Sonnet 4.6. Even if true on your benchmark, that is not representative of the overall performance of the models.

Yes Opus 4.7 fast (no reasoning) did a worst job than Sonnet 4.6 high (with reasoning) according to Gemini 3.1 Pro evaluation.

Your table doesn't indicate reasoning vs non-reasoning, or reasoning level
Post reply on HN