Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

charlesazam.com

Post reply on HN
Print view View original

111–120 of 134 posts

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#111

Post by theptip »

kingkongjaffa wrote: post #102
Huppie wrote: post #75

Earlier quoted context omitted.

Simon Willison recently had a useful tip instructing Claude to use its judgement to pick an appropriate model for tasks. After a short discussion about the idea with Claude mostly on how it fits in my workflow and what models / effort I would like for certain tasks it placed a paragraph in my global Claude.md and it has worked wonders. Ultra became a lot better (faster, cheaper for the same output) and the amount of…

How does it change model mid conversation? Does it spawn sub agents with different models or is it the same single conversation dynamically switching models?

Yes, sub-agents I believe. You need to pay to reload context if you switch the main conversation model effort.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#112

Post by michaellee8 »

cyral wrote: post #81
gmac wrote: post #59

Earlier quoted context omitted.

I've found Opus 4.8 pretty amazing, and Codex a bit meh. YMMV?

I just downloaded and paid for Codex this week because I want to stay on top of the AI tools and understand their capabilities. I've had some good results using 5.6 Sol, although it tends to never want to write any comments (despite modifying the project rules to tell it it MUST), and also occasionally just does a bit of thinking and stops. It'll say "Working through the remaining work" and just ends the chat until I…

I got the complete opposite of what you get, sol ultra literally vibed the entire system out for me from one plan mode approval.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#113

Post by michaellee8 »

Tenoke wrote: post #17

Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.

i found that at 700k-ish context even fable becomes an idiot, maybe openai's decision to cap codex context at 400k is correct, 400k is really a sweet spot where most part of the context is reliable.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#114

Post by jeeeb »

brookst wrote: post #26
jswny wrote: post #19

Earlier quoted context omitted.

Unfortunately I’ve used Claude and GPT models for a long time in a variety of harnesses and I agree with you and I think it’s the compaction. For some reason, codex compaction is like black magic. I’ve never felt like I can just one one continuous thread with other models, Claude I carefully curate when I compact

Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction. Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after. It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude)…

Agree. Another consideration is that input token costs are effectively quadratic with the number of conversation turns. The minute you get into a long conversation you can see the costs shoot up.

You can get far more gas out of even a $20 plan of you’re careful to break things up into relatively small discrete steps, clear context regularly and give the model plenty of information to work with.

My workflow for bigger features is to write out a plan document and then proceed in smaller implementation steps, reviewing as I go. If I find something odd, I ask the agent why and often that leads to discovering a new dimension to the problem, which in turn is an opportunity to adjust the approach.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#115

Post by mdp2021 »

tpicks wrote: post #107
lambda wrote: post #67

Earlier quoted context omitted.

The article describes it. Both Codex and Claude Code have it, but they work slightly differently. Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the main model to indicate what still needs to be done. In Codex, instead it's a tool available to the main model, plus some part of the surrounding harness that will re-prompt it if the…

> Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. feels kinda odd to use a less capable model to determine if the goal is fully complete. Especially if the user is expecting /goal to thoroughly complete the task. A less capable model would be more likely to misclassify `isComplete?`

> odd

There is nothing odd in the basic principle of Economics of using a "good enough" tool, optimal in other respects (e.g. cheap/er).

(You do not hire Nobel Prize winners as receptionists.)

The issue remains, whether the compromise to be adopted actually is "good enough".

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#116

Post by zzleeper »

hereme888 wrote: post #77

GPT-5.6 is unhappy and says the test was materially unfair to the OpenAI model: https://chatgpt.com/c/6a5bbe6a-a760-83ea-931a-4e2bbe028486 Thoughts?

> You don’t have access to this conversation. Make sure you’re logged in to the right account, or ask the conversation owner to send you a share link.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#117

Post by aytigra »

jrockway wrote: post #87
Tenoke wrote: post #17

Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.

I put context size in my Claude Code status bar and ... it does get "tired" when it's at 50%-60% context. I figured this out when I'm like "now modify the testbed to run this and add a test to exercise blah blah blah" and it was just like "That would be a substantial infra lift." In my head I say "I am paying a substantial amount for you to do this!" but just type /compact and re-prompt and my "substantial infra lift…

I've actually found that compacting as often as possible, after planning, then between each implementation step works the best, it both unloads unnecessary context of previous edits and test runs, makes thinking cheaper, and most importantly after each compaction it re-loads CLAUDE.md which make it much more enforcing (otherwise it just moves to the back of context and slips from model's attention).

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#118

Post by fendy3002 »

jrockway wrote: post #87
Tenoke wrote: post #17

Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.

I put context size in my Claude Code status bar and ... it does get "tired" when it's at 50%-60% context. I figured this out when I'm like "now modify the testbed to run this and add a test to exercise blah blah blah" and it was just like "That would be a substantial infra lift." In my head I say "I am paying a substantial amount for you to do this!" but just type /compact and re-prompt and my "substantial infra lift…

Use untracked file as memory and instruct agent to put progress into that, it'll help managing context in current and new sessions

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#119

Post by kaelwd »

esperent wrote: post #24
Tenoke wrote: post #17

Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.

This is one of the advantages of pi. I made a /protect command that protects the message from compaction. I also protect skills automatically. So for long running tasks I'll do /protect your goal is...

https://github.com/k0valik/pi-blackhole works pretty well for this, instead of a summary of a summary of a summary it gets a rolling log of important instructions and discoveries, and can /recall exact contents of a previous message or tool call if needed.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#120

Post by Huppie »

kingkongjaffa wrote: post #102
Huppie wrote: post #75

Earlier quoted context omitted.

Simon Willison recently had a useful tip instructing Claude to use its judgement to pick an appropriate model for tasks. After a short discussion about the idea with Claude mostly on how it fits in my workflow and what models / effort I would like for certain tasks it placed a paragraph in my global Claude.md and it has worked wonders. Ultra became a lot better (faster, cheaper for the same output) and the amount of…

How does it change model mid conversation? Does it spawn sub agents with different models or is it the same single conversation dynamically switching models?

Subagents indeed. This [0] was the post I was referencing, and here's the original suggested prompt:

> For all coding tasks use your judgement to decide an appropriate lower power model and run that in a subagent

As said, I tweaked it a little bit to my preference (mostly because it started delegating a bit too much) and had it placed in my global claude.md but that's the gist.

[0] https://simonwillison.net/2026/Jul/3/judgement/

Post reply on HN