Makes one wonder what the AI was trained with for it to settle on "no means yes if I justify it to myself well enough"
Shall I implement it? No
471–480 of 603 posts
Re: Shall I implement it? No
#472[flagged]
Is this understanding correct: The LLM uses harness tools to ask for permission, then interprets the answer and proceeds. If so, this can't live 100% on the harness. First because you would need the harness to decide when the model should ask for permission or not which is more of an llm-y thing to do. The harness can prevent command executions but wouldn't prevent this case where model goes off and begins reading fi…
Isn't this part of the same problem we have with LLM security in general; that it can only ingest a single stream of tokens, and has no method of privileging "system" tokens over "untrusted" tokens?
If we could solve this (and forgive me if I'm not aware of recent advances that mean we have solved this) then this problem gets easier to solve; permissions live in the system token stream and are privileged. We can then use the LLM to work out what that means in terms of actions.
Re: Shall I implement it? No
#473Another example
I was simply unable to function with Continue in agent mode. I had to switch to chat mode. even tho I told it no changes without my explicit go ahead, it ignored me.
it's actually kind of flabbergasting that the creators of that tool set all the defaults to a situation where your code would get mangled pretty quickly
Re: Shall I implement it? No
#474https://www.infoworld.com/article/4143101/pity-the-developer...
I just wanted to note that the frontier companies are resorting to extreme peer pressure -- and lies -- to force it down our throats
Re: Shall I implement it? No
#475Earlier quoted context omitted.
I worked on a project that did fine tuning and RLHF[1] for a major provider, and you would not believe just how utterly broken a large proportion of the prompts (from real users) were. And the project rules required practically reading tea leaves to divine how to give the best response even to prompts that were not remotely coherent human language. [1] Reinforcement learning from human feedback; basically participant…
I feel like the right response for those situations is to start asking questions of the user. It’s what a human would do if they did not understand.
I made the argument multiple times that the right answer to many prompts would be a question, and it was allowed under some rare circumstances, but far too few.
I suspect in part because the provider also didn't want to create an easy cop out for the people working on the fine-tuning part (a lot of my work was auditing and reviewing output, and there was indeed a lot of really sloppy work, up to and including cut and pasting output from other LLMs - we know, because on more than one occasion I caught people who had managed to include part of Claudes website footer in their answer...)
Re: Shall I implement it? No
#476Earlier quoted context omitted.
Yeah, anyone who’s used LLMs for a while would know that this conversation is a lost cause and the only option is to start fresh. But, a common failure mode for those that are new to using LLMs, or use it very infrequently, is that they will try to salvage this conversation and continue it. What they don’t understand is that this exchange has permanently rotted the context and will rear its head in ugly ways the long…
I like how anything these tools do wrong just boils down to “you’re using it wrong” It can do no wrong It is unfalsifiable as a tool
I don't think it's intended as that kind of binary. It's more like "yeah, it's flawed in that way, and here's how you can get around that". If someone's claiming the tool is perfect, they're wrong; but if someone's repeatedly using it in the way that doesn't work and claiming the tool is useless, they're also wrong.
Re: Shall I implement it? No
#477Codex has always been better at following agents.md and prompts more, but I would say in the last 3 months both Claude Code got worse (freestyling like we see here) and Codex got EVEN more strict. 80% of the time I ask Claude Code a question, it kinda assumes I am asking because I disagree with something it said, then acts on a supposition. I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT COD…
I tried using codex, and it is great (meaning - boring) when it works. My problem is it does not work. Let me explain
codex> Next I can make X if you agree.
me> ok
codex> I will make X now
me> Please go on
codex> Great, I am starting to work on X now
me> sure, please do
codex> working on X, will report on completion
me> yo good? please do X!
... and so on. Sometimes one round, sometimes four, plus it stops after every few lines to "report progress" and needs another nudge or five. :(
Re: Shall I implement it? No
#478Just yesterday I had a moment Claude's code in a conversation said - “Yes. I just looked at tag names and sorted them by gut feeling into buckets. No systematic reasoning behind it.” It has gut feelings now? I confronted for a minute - but pulled out. I walked away from my desk for an hour to not get pulled into the AInsanity.
Even when used by humans, "gut feelings" is still a metaphor.
Re: Shall I implement it? No
#479Codex has always been better at following agents.md and prompts more, but I would say in the last 3 months both Claude Code got worse (freestyling like we see here) and Codex got EVEN more strict. 80% of the time I ask Claude Code a question, it kinda assumes I am asking because I disagree with something it said, then acts on a supposition. I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT COD…
First time I used Claude I asked it to look at the current repo and just tell me where the database connection string was defined. It added 100 lines of code.
I asked it to undo that and it deleted 1000 lines and 2 files
Re: Shall I implement it? No
#480Codex has always been better at following agents.md and prompts more, but I would say in the last 3 months both Claude Code got worse (freestyling like we see here) and Codex got EVEN more strict. 80% of the time I ask Claude Code a question, it kinda assumes I am asking because I disagree with something it said, then acts on a supposition. I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT COD…
“Don’t code yet” is a longstanding part of the rapport