Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6

openai.com

Post reply on HN
Print view View original

981–990 of 1001 posts

Re: GPT-5.6

#981

Post by pimeys »

enraged_camel wrote: post #965
pimeys wrote: post #905

I've been testing Sol/Terra/Luna now since yesterday, running complex evals on all of them and I feel a bit... mixed on how they perform. The eval is an agent that runs a set of tools and a prompt we can tune separately for different models. The OpenAI version of the prompt was specifically tuned based on their guide[0]. Then we let Opus to run another agent that acts as a user, trying to solve a problem (anonymized…

Yes, this is exactly why I said yesterday that OpenAI does benchmaxxing, seemingly quite a bit [1]. I got a flurry of downvotes for it at first, but I think people came around to it once they tried the model like you did. Ultimately I think the issue is that OpenAI is under tremendous pressure to perform, but GPT-6 is not ready yet, so they had to push GPT-5 to its limits, and the only way they could do it was with r…

For me the weirdest is Luna. It costs the same as GLM to solve a task with it, but it just calls the same failing tool over and over again until we cut it out.

Now if you look at where GLM stands, or even DeepSeek v4 Flash, things get really interesting for what they provide.

US labs completely miss a cheap model that can solve problems for 95% of the people. Gemini 3.5 Flash could've been it if it didn't burn so many tokens.

Re: GPT-5.6

#982

Post by arendtio »

I also like to take a look at https://cursor.com/cursorbench

While in the past months Composer 2.5 was a lot better than I had expected a year ago, and the GPT 5.6 family does a good job in terms of cost for performance, I wonder why nobody is talking about Grok 4.5 high? Those numbers look very convincing to me.

Re: GPT-5.6

#983

Post by m0rde »

arendtio wrote: post #982

I also like to take a look at https://cursor.com/cursorbench While in the past months Composer 2.5 was a lot better than I had expected a year ago, and the GPT 5.6 family does a good job in terms of cost for performance, I wonder why nobody is talking about Grok 4.5 high? Those numbers look very convincing to me.

> * Grok 4.5 has an advantage on CursorBench: an earlier snapshot of the Cursor codebase was unintentionally included in training. The exact score impact is unclear. That data has been removed for future models. For a rundown of third-party benchmark scores, see the Grok 4.5 launch blog.

I don't know about those numbers, even assuming this was by mistake :)

Re: GPT-5.6

#984

Post by ai_fry_ur_brain »

cognitiveinline wrote: post #970
ai_fry_ur_brain wrote: post #817

Earlier quoted context omitted.

The literal worst time. I prefer meritocracies.

You think aristrocrats are using GPT and succeeding?

No I think aristocrats are using llms to justify the flattening of wages, where they can use it as an excuse diminish the value of merit based systems.. Meritocracy / education was one of the few ways someone from a lower class couod climb the ladder and they're trying to destroy that ladder.

Re: GPT-5.6

#985

Post by killix »

steve-atx-7600 wrote: post #452
Syntaf wrote: post #6

Ok long time Claude Code user here; lately I've started to realize there's other great models out there I should be trying, but I'm hesitant to leave Claude Code behind for something new. What's the consensus today on codex vs claude code, does it really matter anymore?

Set yourself up to be able to try / switch between models easily. I was a claude only user and just have my user level AGENTS.md for codex and others simply point at my user CLAUDE.md. Have a script that syncs my skills (just directories) between all models. Also, if you want to use /simplify or similar from claude in another model, you can ask claude for the prompt and put that in a skill for the other models.

[flagged]

Re: GPT-5.6

#986

Post by dboreham »

gruntled-worker wrote: post #311
dboreham wrote: post #224

Earlier quoted context omitted.

Nudged by this thread, I've decided to switch from Claude to Codex for a bit to see what happens. But...I immediately became lost in their marketing vortex of confusion on plans and pricing. Anyone care to tell me which plan I should be using? On the other side I use the $100 Claude Code plan. We actually have a "Business" ChatGPT subscription already, which seems to be $50/mo/seat. OpenAI's web site offers a set of…

Test-drive it with an individual Pro account (5x or 20x) for a month. Download the Codex CLI client from https://github.com/openai/codex and auth it in the browser via the URL it provides. Set the model to 5.6-Sol and effort to max.

Ok thanks. Executed on that. I had it build a simple project (actually not so simple since it involved domain knowledge about snowpack and ice) and also gave the same prompt to Opus and to Fable. When I get some spare time I'll write an article highlighting the differences between all three.

Re: GPT-5.6

#987

Post by versteegen »

davidpapermill wrote: post #875
navigate8310 wrote: post #515

Earlier quoted context omitted.

There goes my plan to buy a PC for the next decade

I actually think we're in a strange situation with AI compute. Right now, we have models that are statistical models of language, with a world model and reasoning "falling out" of a lot of effort. It's like we've made something that's a little bit intelligent, and now we're trying to amplify that trick to create something that's quite intelligent. And - don't get me wrong - it works. But it's also super, super ineffi…

I agree. But I think you're missing that LLMs can internalise a lot of the thinking process in their layers without explicit CoT. That System 1-style reasoning is bounded depth computation but very, very broad. Yudkowsky called it "cached thoughts" and I think it's an incredibly important idea [1]. It's really stiking how the best LLMs don't even need to think where smaller LLMs do.

So as more thinking is cached in their weights through increased RL training, those weights are doing more useful work and the efficiency is increasing.

[1] https://www.lesswrong.com/posts/2MD3NMLBPCqPfnfre/cached-tho...

Re: GPT-5.6

#988

Post by [deleted] »

vatsachak wrote: post #246
10xDev wrote: post #234

Earlier quoted context omitted.

Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.

I mean, theoretically you can solve every finitary problem with a brute force solution... Richard Sutton specifically states that the search has to be smart. We know that the brain uses recurrent connections and is shallow. I think a lot more money has to go into architecture. Feed Forward transformers can only scale so far

[deleted]

Re: GPT-5.6

#989

Post by dyauspitr »

Scoundreller wrote: post #937
dyauspitr wrote: post #508

Earlier quoted context omitted.

Honestly it’s the usage limits that are so generous that makes codex worth it even if it may not be exactly as powerful as Claude. The peace of mind that you can try a lot of things and make huge refactors and run extensive redundant tests without running out of tokens just makes the whole thing a much better experience. I tried coding with Deepseek and it was pretty terrible so the only reason codex works is because…

> I tried coding with Deepseek But soooooo cheap. Especially for those of us where a monthly sub doesn’t make sense.

There almost no scenario in which the sub doesn’t make sense unless you write code once a week or something. Even then you’re better off with the lowest tier $20 plan.

Re: GPT-5.6

#990

Post by camillomiller »

andriy_koval wrote: post #968
camillomiller wrote: post #899

Earlier quoted context omitted.

I would add to this that, to me and to many friends in their 30-40s, using AI models to achieve something we used our brains to achieve feels... empty? wrong? soulless? Sure, a lot of menial work can be relegated to the models, and it's ok, most of the time, but you finish a day of work with the inability to shake a precise new feeling: that you haven't really achieved something, even if you shipped more than on an a…

> to me and to many friends in their 30-40s, using AI models to achieve something we used our brains to achieve feels... empty? you have to start using AI to achieve something which was way more challenging before, then you will feel lots of fulfillment and inspiration.

But this is the issue I have. I will have removed all desirable difficulties from that harder endeavor and I will have learned very little, if nothing
Post reply on HN