Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shall I implement it? No

gist.github.com

Post reply on HN
Print view View original

121–130 of 603 posts

Re: Shall I implement it? No

#121

Post by piiritaja »

Hansenq wrote: post #96

Often times I'll say something like: "Can we make the change to change the button color from red to blue?" Literally, this is a yes or no question. But the AI will interpret this as me _wanting_ to complete that task and will go ahead and do it for me. And they'll be correct--I _do_ want the task completed! But that's not what I communicated when I literally wrote down my thoughts into a written sentence. I wonder wh…

I mean humans communicate the same way. We don't interpret the words literally and neither does the LLM. We think about what one is trying to communicate to the other.

For example If you ask someone "can you tell me what time it is?", the literal answer is either "yes"/"no". If you ask an LLM that question it will tell you the time, because it understands that the user wants to know the time.

Re: Shall I implement it? No

#122

Post by reconnecting »

sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

There is the link to the full session below.

https://news.ycombinator.com/item?id=47357042#47357656

Re: Shall I implement it? No

#124

Post by BosunoB »

sgillen wrote: post #74

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched fr…

The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it.

The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.

Re: Shall I implement it? No

#125

Post by Aeolun »

alpb wrote: post #85

I see on a daily basis that I prevent Claude Code from running a particular command using PreToolUse hooks, and it proceeds to work around it by writing a bash script with the forbidden command and chmod+x and running it. /facepalm

Maybe that means you need to change the text that comes out of the pre hook?

Re: Shall I implement it? No

#126

Post by singron »

This is very funny. I can see how this isn't in the training set though.

1. If you wanted it to do something different, you would say "no, do XYZ instead".

2. If you really wanted it to do nothing, you would just not reply at all.

It reminds me of the Shell Game podcast when the agents don't know how to end a conversation and just keep talking to each other.

Re: Shall I implement it? No

#127

Post by prmph »

XCSme wrote: post #16

Claude is quite bad at following instructions compared to other SOTA models. As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."

They all are. And once the context has rotted or been poisoned enough, it is unsalvageable.

Claude is now actually one of the better ones at instruction following I daresay.

Re: Shall I implement it? No

#128

Post by XCSme »

wouldbecouldbe wrote: post #31
XCSme wrote: post #16

Claude is quite bad at following instructions compared to other SOTA models. As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."

I think its why its so good; it works on half ass assumptions, poorly written prompts and assumes everything missing.

To be honest, I had this "issue" too.

I upgraded to a new model (gpt-4o-mini to grok-4.1-fast), suddenly all my workflows were broken. I was like "this new model is shit!", then I looked into my prompts and realized the model was actually better at following instructions, and my instructions were wrong/contradictory.

After I fixed my prompts it did exactly what I asked for.

Maybe models should have another tuneable parameters, on how well it should respect the user prompt. This reminds me of imagegen models, where you can choose the config/guidance scale/diffusion strength.

Re: Shall I implement it? No

#130

Post by xeromal »

bjackman wrote: post #89

I have also seen the agent hallucinate a positive answer and immediately proceed with implementation. I.e. it just says this in its output: > Shall I go ahead with the implementation? > Yes, go ahead > Great, I'll get started.

I love when mine congratulates itself on a job well-done
Post reply on HN