I guess Plus accounts don't get access to Sol? Or is it because I am in Europe?
GPT-5.6
461–470 of 1001 posts
Re: GPT-5.6
#462Re: GPT-5.6
#463Here are 18 pelicans - six each for Luna, Terra and Sol at the six different reasoning effort levels (plus the price to generate each one): https://static.simonwillison.net/static/2026/gpt-5.6-pelican... Or if you want to see some in 3D, OpenAI featured a pelican riding a tricycle, bicycle, pony and another pelican in their livestream this morning: https://www.youtube.com/live/Wq45rvPGNHs?t=1070s
[flagged]
Re: GPT-5.6
#464The meat of the report for SWEs: SWE-Bench Pro Sol: 64.6% Fable: 80% Opus: 69.2% (!!!!) So, it still trails Opus, significantly, and is not a next-gen coding model like Mythos/Fable 5. Disappointing to say the least, but somewhat expected.
You've overstated the conclusion. The SWE-bench series has had issues since its inception. OpenAI no longer recommends SWE-Bench-Pro as a benchmark: https://openai.com/index/separating-signal-from-noise-coding...
That's smart, they only recommend benchmarks that make them look better then their competitors.
Re: GPT-5.6
#465Here are 18 pelicans - six each for Luna, Terra and Sol at the six different reasoning effort levels (plus the price to generate each one): https://static.simonwillison.net/static/2026/gpt-5.6-pelican... Or if you want to see some in 3D, OpenAI featured a pelican riding a tricycle, bicycle, pony and another pelican in their livestream this morning: https://www.youtube.com/live/Wq45rvPGNHs?t=1070s
At what stages will models start to internally reflect the drawn SVG and automatically fix their own mistakes? I assume multimodal models can do it already do it today if constantly asked "make it better"
I haven't tried this in a few months, but last time I tried a loop that rendered the pelican and asked for improvements the results were actually quite disappointing. Be interesting to try that again against GPT-5.6 at Claude Fable 5 though.
Re: GPT-5.6
#466Earlier quoted context omitted.
It seems like the way brevity instructions have changed is mis-aligned with how most people would expect to use them or are currently using them. Here's the example they give: > Instead of asking for the shortest possible answer, replace brevity instructions with prioritization: > Lead with the conclusion. Include the evidence needed to support it, any material caveat, and the next action. Omit secondary detail and r…
> Lead with conclusion. I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion. Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberati…
I wonder if it would help to ask it to write a rough draft and then reorder it?
Re: GPT-5.6
#467Not specific to OpenAI / Codex, but I'm curious what people are doing to protect themselves from any destructive actions by their coding agents? Just install and pray? Explicity approve all actions? Reconfigure for safety? Run in a sandbox (Docker) ?
Re: GPT-5.6
#468Here are 18 pelicans - six each for Luna, Terra and Sol at the six different reasoning effort levels (plus the price to generate each one): https://static.simonwillison.net/static/2026/gpt-5.6-pelican... Or if you want to see some in 3D, OpenAI featured a pelican riding a tricycle, bicycle, pony and another pelican in their livestream this morning: https://www.youtube.com/live/Wq45rvPGNHs?t=1070s
Cool. I still find these a useful visualization of some the qualities of llms. Even if they did train for [animal] on [vehicle] svg, it's still nice to see at a glance how the different models and reasoning levels perform. Lunar misses part of the frame, except on max reasoning. While most of the others have a mostly correct bike at all reasoning levels.
I once used something like karpathy's auto-scientist to mutate the prompts and rank them with a vison model. Some of the winners where pretty neat. I think they have a lot more style than the gpt-5.6 ones. https://xcancel.com/xundecidability/status/20449185674144196...
Re: GPT-5.6
#469Not specific to OpenAI / Codex, but I'm curious what people are doing to protect themselves from any destructive actions by their coding agents? Just install and pray? Explicity approve all actions? Reconfigure for safety? Run in a sandbox (Docker) ?
I still just explicitly approve all actions and review all code (unless it's a personal/throwaway project no one else will ever touch/use/see). I know a lot of people that run in a sandbox though. That said, I'm sure there are lots of people that just yolo it and hope for the best.
Re: GPT-5.6
#470The developer's guide ( https://developers.openai.com/api/docs/guides/latest-model ) has some interesting semantic tips for using the model: > Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly. > Original image detail: GPT-5.6 preserves the…
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.” What about my favorite, "no yapping"?
It might need the longer answer to think about the question, so one approach would be to ask it normally and then ask it to repeat itself shorter.