421–430 of 603 posts
Never trust a LLM for anything you care about.
never trust a screenshot of a command prompts output blindly either. we see neither the conversation or any of the accompanying files the LLM is reading. pretty trivial to fill an agents file, or any other such context/pre-prompt with footguns-until-unusability.
[flagged]
I know, it's not really an appropriate use of the tool, but I'm a lazy programmer and used what I had ready access to. And it took like 5 iterations.
Discrete, concrete things like "stop", or "no" is just like... not in its wheelhouse.
This is very funny. I can see how this isn't in the training set though. 1. If you wanted it to do something different, you would say "no, do XYZ instead". 2. If you really wanted it to do nothing, you would just not reply at all. It reminds me of the Shell Game podcast when the agents don't know how to end a conversation and just keep talking to each other.
Yes = do it
No = don‘t do it
Earlier quoted context omitted.
> I consulted Claude chat and it admitted this as a major problem with Claude these days, and suggested that I should ask what are the coordinates of UI controls are on screenshot thus forcing it to look If 3 years into LLMs even HNers still don't understand that the response they give to this kind of question is completely meaningless, the average person really doesn't stand a chance.
The whole “chat with an AI” paradigm is the culprit here. Priming people to think they are actually having a conversation with something that has a mind model. It’s just a text generator that generates plausible text for this role play. But the chat paradigm is pretty useful in helping the human. It’s like chat is a natural I/O interface for us.
I’m not an active LLMs user, but I was in a situation where I asked Claude several times not to implement a feature, and that kept doing it anyway.
Yeah, anyone who’s used LLMs for a while would know that this conversation is a lost cause and the only option is to start fresh. But, a common failure mode for those that are new to using LLMs, or use it very infrequently, is that they will try to salvage this conversation and continue it. What they don’t understand is that this exchange has permanently rotted the context and will rear its head in ugly ways the long…
It can do no wrong
It is unfalsifiable as a tool
Earlier quoted context omitted.
How would it know not to ask again if it can't store a cookie?
At least if this "Store cookies?" question is implicitly referencing EU regulations, those regulations don't require consent for cookies which are considered essential, including a cookie to store the response to the consent question (but certainly not advertising tracking cookies). So the respectful replacement for "Ask me again" is "Essential cookies only" (or some equivalent wording to "Essential" like "Required"…
Its gotten so bad that Claude will pretend in 10 of 10 cases that task is done/on screenshot bug is fixed, it will even output screenshot in chat, and you can see the bug is not fixed pretty clear there. I consulted Claude chat and it admitted this as a major problem with Claude these days, and suggested that I should ask what are the coordinates of UI controls are on screenshot thus forcing it to look. So I did that…
> I consulted Claude chat and it admitted this as a major problem with Claude these days, and suggested that I should ask what are the coordinates of UI controls are on screenshot thus forcing it to look If 3 years into LLMs even HNers still don't understand that the response they give to this kind of question is completely meaningless, the average person really doesn't stand a chance.
Earlier quoted context omitted.
Mine's started to use $() to feed e.g. strings into a commit. Because this is a command expansion it requires approval every single time.
Yeah, mine to which I find really annoying
Codex has always been better at following agents.md and prompts more, but I would say in the last 3 months both Claude Code got worse (freestyling like we see here) and Codex got EVEN more strict. 80% of the time I ask Claude Code a question, it kinda assumes I am asking because I disagree with something it said, then acts on a supposition. I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT COD…