Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

Post by lionkor » Sat, Jul 18, 2026, 7:10 PM UTC

throw10920 wrote: post #89

lionkor wrote: post #83

Earlier quoted context omitted.

It's part of the rationale for why one would choose Pi over Claude, even if Pi lacks the feature. I admit my tone is entirely inappropriate though.

> It's part of the rationale for why one would choose Pi over Claude It's unrelated to my direct comment and it's a generic tangent off the thread further up, which is about technical issues with Claude Code. As the guidelines say: "On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratif…

Post by theptip » Sat, Jul 18, 2026, 7:31 PM UTC

ttul wrote: post #71

theptip wrote: post #61

Great eval! If you are comparing search strategies, ultra mode is likely superior. Would love to see a follow-up evaluating that. Ultra can fan out parallel investigators, run adversarial review at defined checkpoints, and do a bunch of other smart stuff to avoid getting stuck in a local optimum. Generally as the OP notes, /goal works better for single-track investigations or small scale scatter/gather.

I think ultra mode needs to be more clearly documented (or perhaps cautioned against!). Most devs - myself included - who saw “ultra” mode figured that it’s just a magic bullet that makes the model work harder and achieve better results. But, for many tasks, ultra mode is possibly worse and certainly more expensive.

Post by mh- » Sat, Jul 18, 2026, 7:52 PM UTC

jascha_eng wrote: post #63

madeofpalk wrote: post #49

Earlier quoted context omitted.

Adding a new lint rule that a lot of code violates.

And that doesn't work with a simple prompt?

Post by irthomasthomas » Sat, Jul 18, 2026, 8:27 PM UTC

kccqzy wrote: post #78

brookst wrote: post #26

Earlier quoted context omitted.

Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction. Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after. It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude)…

You just haven’t worked on tasks that are complicated enough. Occasionally it took more than 1M tokens just to come up with a plausible plan. Personally I find using /rewind judiciously is better than using /compact. The latter essentially gives you no control of what details to discard, but the former at least has coarse-grained control.

Post by brookst » Sat, Jul 18, 2026, 9:02 PM UTC

wgjordan wrote: post #76

brookst wrote: post #26

Earlier quoted context omitted.

Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction. Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after. It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude)…

> With 1m context window there is no reason for a single task to require compaction Only if money is no object. Cache reads are cheap (10% of uncached input costs) but definitely not free, and cached reads dominate session costs at long context lengths. A prompt at 20k context with $0.01 in cached reads would cost $0.40 in cached reads at 800k context, that quickly adds up for long sessions.