Live data from Hacker News

Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

charlesazam.com

41–50 of 134 posts

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#41
post #3

The chart at the top is somewhat confusing. It says, “lower is better” but the y-axis is inverted! So visually higher in the chart is better but lower in terms of # value.

It isn’t inverted though? 32000 is at the bottom, 40000 is at the top. Maybe it changed since you commented.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#42

What is /goal?

An agent is an llm running on a loop until it decides it should stop.

/goal is a gimmick where you run a "parent" agent on top that runs the agent on a loop until the it decides to stop, just prompting it "nope, not done yet, continue".

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#43

/goal has replaced plan mode for me. This is the pattern I use for 95% of my AI work now: 1. Read X feature of Y and tell me when you fully understand it (if there's any detail missing in the summary, repeat until the context is primed) 2. What time is it? 3. /goal Spend X minutes from $time writing a technical design doc on $feature. There must not be any vague language or ambiguity in the document. Read carry_forwa…

> 3. /goal Spend X minutes from $time writing a technical design doc on $feature. Hmm, I feel like this is akin to making a recursive function have a exit condition not based on what it actually did/found, but based on how long time it took. I'm always using /goal with explicit goals that the agent needs to achieve. Time-bounding them wouldn't make sense, I want something specific done regardless of how long time it…

I think like the other commentator noted, I don't do this blindly, as I also wouldn't if I was instructing a junior engineer. I have a rough sense of how long it would take me to research all angles of the task. 5 minutes for a complicated feature is too short for a comprehensive plan. Letting it run for a few hours is also probably unproductive. But 10-60 minutes depending on complexity with followup review is a nice sweet spot. Just my $0.02.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#44

I love that we have this on one hand and me cleaning up catastrophic CSS made by Sol on the other. Then again, maybe CSS is the ultimate benchmark.

Codex has a very bad system prompt which no one at OpenAI is seemingly aware of.

Sol frontend is perfectly serviceable if you use it from something like Pi.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#46
post #26
post #19

Earlier quoted context omitted.

Unfortunately I’ve used Claude and GPT models for a long time in a variety of harnesses and I agree with you and I think it’s the compaction. For some reason, codex compaction is like black magic. I’ve never felt like I can just one one continuous thread with other models, Claude I carefully curate when I compact

Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction. Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after. It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude)…

Most models’ reasoning abilities drops off significantly between the 256K-1M token ranges of the context window. There’s too much stuff to “pay attention to” at once.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#49

Earlier quoted context omitted.

It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.

Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?

Adding a new lint rule that a lot of code violates.

Re: Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

#50

...is this not a Travelling Salesman Problem?

Related, but not quite the same. The solution to this problem can have more than one loop, whereas TSP definitionally has only one. Not quite sure how that changes the difficulty or the tools that you’d use.
Post reply on HN