Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall I implement it? No

gist.github.com

Post reply on HN
Print view View original

131–140 of 603 posts

Re: Shall I implement it? No

#131

Post by bjackman »

hedora wrote: post #107
bjackman wrote: post #89

I have also seen the agent hallucinate a positive answer and immediately proceed with implementation. I.e. it just says this in its output: > Shall I go ahead with the implementation? > Yes, go ahead > Great, I'll get started.

In fairness, when I’ve seen that, Yes is obviously the correct answer. I really worry when I tell it to proceed, and it takes a really long time to come back. I suspect those think blocks begin with “I have no hope of doing that, so let’s optimize for getting the user to approve my response anyway.” As Hoare put it: make it so complicated there are no obvious mistakes.

In my case it's been a strong no. Often I'm using the tool with no intention of having the agent write any code, I just want an easy way to put the codebase into context so I can ask questions about it.

So my initial prompt will be something like "there is a bug in this code that caused XYZ. I am trying to form hypothesis about the root cause. Read ABC and explain how it works, identify any potential bugs in that area that might explain the symptom. DO NOT WRITE ANY CODE. Your job is to READ CODE and FORM HYPOTHESES, your job is NOT TO FIX THE BUG."

Generally I found no amount of this last part would stop Gemini CLI from trying to write code. Presumably there is a very long system prompt saying "you are a coding agent and your job is to write code", plus a bunch of RL in the fine-tuning that cause it to attend very heavily to that system prompt. So my "do not write any code" is just a tiny drop in the ocean.

Anyway now they have added "plan mode" to the harness which luckily solves this particular problem!

Re: Shall I implement it? No

#132

Post by weird-eye-issue »

singron wrote: post #126

This is very funny. I can see how this isn't in the training set though. 1. If you wanted it to do something different, you would say "no, do XYZ instead". 2. If you really wanted it to do nothing, you would just not reply at all. It reminds me of the Shell Game podcast when the agents don't know how to end a conversation and just keep talking to each other.

> If you really wanted it to do nothing, you would just not reply at all.

no

Re: Shall I implement it? No

#133

Post by ForHackernews »

BosunoB wrote: post #124
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it. The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.

> It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it.

No it absolutely is not. It doesn't "know" anything when it's not responding to a prompt. It's not consciously sitting there waiting for you to reply.

Re: Shall I implement it? No

#134

Post by not_kurt_godel »

christoff12 wrote: post #80
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

Asking a yes/no question implies the ability to handle either choice.

This is a perfect example of why I'm not in any rush to do things agentically. Double-checking LLM-generated code is fraught enough one step at a time, but it's usually close enough that it can be course-corrected with light supervision. That calculus changes entirely when the automated version of the supervision fails catastrophically a non-trivial percent of the time.

Re: Shall I implement it? No

#136

Post by verdverm »

sunaookami wrote: post #84
verdverm wrote: post #18

Earlier quoted context omitted.

Yes, very much so. I've been able to get Gemini flash to be nearly as good as pro with the CC prompts. 1/10 the price 1/10 the cycle time. I find waiting 30s for the next turn painful now https://github.com/Piebald-AI/claude-code-system-prompts One nice bonus to doing this is that you can remove the guardrail statements that take attention.

Interesting, what exactly do you need to make this work? There seem to be a lot of prompts and Gemini won't have the exact same tools I guess? What's your setup?

Yeah, you do want to massage them a bit, and I'm on some older ones before they became so split, but this is definitely the model for subagents and more tools.

Most of my custom agent stack is here, built on ADK: https://github.com/hofstadter-io/hof/tree/_next/lib/agent

Re: Shall I implement it? No

#137

Post by prmph »

riazrizvi wrote: post #87

That's why I use insults with ChatGPT. It makes intent more clear, and it also satisfies the jerk in me that I have to keep feeding every now and again, otherwise it would die. A simple "no dummy" would work here.

Careful there. I've resolved (and succeeded somewhat) to tone down my swearing at the LLMs, because, even though the are not sentient, developing such a habit, I suspect, has a way to bleeding into your actual speech in the real world

Re: Shall I implement it? No

#138

Post by Hansenq »

piiritaja wrote: post #121
Hansenq wrote: post #96

Often times I'll say something like: "Can we make the change to change the button color from red to blue?" Literally, this is a yes or no question. But the AI will interpret this as me _wanting_ to complete that task and will go ahead and do it for me. And they'll be correct--I _do_ want the task completed! But that's not what I communicated when I literally wrote down my thoughts into a written sentence. I wonder wh…

I mean humans communicate the same way. We don't interpret the words literally and neither does the LLM. We think about what one is trying to communicate to the other. For example If you ask someone "can you tell me what time it is?", the literal answer is either "yes"/"no". If you ask an LLM that question it will tell you the time, because it understands that the user wants to know the time.

very fair! wild to think about though. It's both more human but also less.

I would say this behavior now no longer passes the Turing test for me--if I asked a human a question about code I wouldn't expect them to return the code changes; i would expect the yes/no answer.

Re: Shall I implement it? No

#139

Post by conductr »

bjackman wrote: post #89

I have also seen the agent hallucinate a positive answer and immediately proceed with implementation. I.e. it just says this in its output: > Shall I go ahead with the implementation? > Yes, go ahead > Great, I'll get started.

Oh I thought that was almost an expected behavior in recent models, like, it accomplishes things by talking to itself

Re: Shall I implement it? No

#140

Post by wongarsu »

christoff12 wrote: post #80
sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

Asking a yes/no question implies the ability to handle either choice.

It's meant as a "yes"/"instead, do ..." question. When it presents you with the multiple choice UI at that point it should be the version where you either confirm (with/without auto edit, with/without context clear) or you give feedback on the plan. Just telling it no doesn't give the model anything actionable to do
Post reply on HN