Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall I implement it? No

gist.github.com

Post reply on HN
Print view View original

141–150 of 603 posts

Re: Shall I implement it? No

#141

Post by Lerc »

christoff12 wrote: post #80
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

Asking a yes/no question implies the ability to handle either choice.

But I think if you sit down and really consider the implications of it and what yes or not actually means in reality, or even a overabundance of caution causing extraneous information to confuse the issue enough that you don't realise that this sentence is completely irrelevant to the problem at hand and could be inserted by a third party, yet the AI is the only one to see it. I agree.

Re: Shall I implement it? No

#142

Post by efitz »

christoff12 wrote: post #80
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

Asking a yes/no question implies the ability to handle either choice.

To an LLM, answering “no” and changing the mode of the chat window are discrete events that are not necessarily related.

Many coding agents interpret mode changes as expressions of intent; Cline, for example, does not even ask, the only approval workflow is changing from plan mode to execute mode.

So while this is definitely both humorous and annoying, and potentially hazardous based on your workflow, I don’t completely blame the agent because from its point of view, the user gave it mixed signals.

Re: Shall I implement it? No

#143

Post by bensyverson »

reconnecting wrote: post #122
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

There is the link to the full session below. https://news.ycombinator.com/item?id=47357042#47357656

Do we know if thinking was on high effort? I've found it sometimes overthinks on high, so I tend to run on medium.

Re: Shall I implement it? No

#144

Post by inerte »

Codex has always been better at following agents.md and prompts more, but I would say in the last 3 months both Claude Code got worse (freestyling like we see here) and Codex got EVEN more strict.

80% of the time I ask Claude Code a question, it kinda assumes I am asking because I disagree with something it said, then acts on a supposition. I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT CODE. DO NOT RUN COMMANDS". Which is ridiculous.

Codex, on the other hand, will follow something I said pages and pages ago, and because it has a much larger context window (at least with the setup I have here at work), it's just better at following orders.

With this project I am doing, because I want to be more strict (it's a new programming language), Codex has been the perfect tool. I am mostly using Claude Code when I don't care so much about the end result, or it's a very, very small or very, very new project.

Re: Shall I implement it? No

#145

Post by keerthiko »

wongarsu wrote: post #140
christoff12 wrote: post #80

Earlier quoted context omitted.

Asking a yes/no question implies the ability to handle either choice.

It's meant as a "yes"/"instead, do ..." question. When it presents you with the multiple choice UI at that point it should be the version where you either confirm (with/without auto edit, with/without context clear) or you give feedback on the plan. Just telling it no doesn't give the model anything actionable to do

It can terminate the current plan where it's at until given a new prompt, or move to the next item on its todo list /shrug

Re: Shall I implement it? No

#146

Post by XCSme »

prmph wrote: post #127
XCSme wrote: post #16

Claude is quite bad at following instructions compared to other SOTA models. As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."

They all are. And once the context has rotted or been poisoned enough, it is unsalvageable. Claude is now actually one of the better ones at instruction following I daresay.

In my tests it's worst with adding extra formatting or output: https://aibenchy.com/compare/anthropic-claude-opus-4-6-mediu...

For example, sometimes it outputs in markdown, without being asked to (e.g. "**13**" instead of "13"), even when asked to respond with a number only.

This might be fine in a chat-environment, but not in a workflow, agentic use-case or tool usage.

Yes, it can be enforced via structured output, but in a string field from a structured output you might still want to enforce a specific natural-language response format, which can't be defined by a schema.

Re: Shall I implement it? No

#147

Post by inerte »

xeromal wrote: post #130
bjackman wrote: post #89

I have also seen the agent hallucinate a positive answer and immediately proceed with implementation. I.e. it just says this in its output: > Shall I go ahead with the implementation? > Yes, go ahead > Great, I'll get started.

I love when mine congratulates itself on a job well-done

Mine on Plan Mode sometimes says "Excellent research!" (of course to the discovery it just did)

Re: Shall I implement it? No

#149

Post by stefan_ »

sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

This is probably just OpenCode nonsense. After prompting in "plan mode", the models will frequently ask you if you want to implement that, then if you don't switch into "build mode", it will waste five minutes trying but failing to "build" with equally nonsense behavior.

Honestly OpenCode is such a disappointment. Like their bewildering choice to enable random formatters by default; you couldn't come up with a better plan to sabotage models and send them into "I need to figure out what my change is to commit" brainrot loops.

Re: Shall I implement it? No

#150

Post by nulltrace »

I've seen something similar across Claude versions.

With 4.0 I'd give it the exact context and even point to where I thought the bug was. It would acknowledge it, then go investigate its own theory anyway and get lost after a few loops. Never came back.

4.5 still wandered, but it could sometimes circle back to the right area after a few rounds.

4.6 still starts from its own angle, but now it usually converges in one or two loops.

So yeah, still not great at taking a hint.

Post reply on HN