To remedy this I’m working on the /beat command, which will simulate you (the user) beating up the agent. Excited for my new career in AI ethics!
The user is visibly frustrated
71–80 of 288 posts
Re: The user is visibly frustrated
#72Earlier quoted context omitted.
Semi-related, I'm always very put off by how people treat LLMs. Especially coders, seems an instinctive joy comes out to play God. The justification is usually that it's intentionally against the trap of anthropomorphizing, but no I can't help but suspect it's people getting off on power. It's weird. I am always very cordial in my sessions. It's just more pleasant and it's a habit I want to habituate. Great work! Now…
The similarity to human interaction is irrelevant. No one apologises to a potato being peeled, nor compliments it for doing a great job being mashed.
I'm willing to bet you've never sent a single word to a potato ever. And you send thousands to an llm.
This is not about llm sentience. this is about the habit and skill of communication.
Re: The user is visibly frustrated
#73For me, LLMs tend to engage the 'language center' that drains me faster than the 'problem solving center' I usually reserve for writing code. We really need a different abstraction the bridges the gap between human and programming language, and load balance between these two parts of the brain more effectively.
I've been thinking recently of creating a programming language where you write mostly python, but can just "hand wave" away the boring stuff for the agent to do. If you don't want to deal with it, just type in a prompt or pseudocode and it will get filled in. Kinda like using the ai-assisted image editing software.
the main difference being that you don't switch between an agent chat window and the code. Just leave a note to the agent and go back to coding as usual, while the agent fills in the gap.
Re: The user is visibly frustrated
#74Its kind of astonishing to see years of traditional software engineering practices being tossed aside in the rush for the Latest Cool New Thing™ ... have people really forgotten that you have to apply a workflow to software development, in order to have quality software?
You don't just write it, compile it, run it and ship it - do you? Surely, in the rush to become as agile as possible, folks haven't forgotten their quality checks in the workflow/process?
I have had great success with AI coding these days .. but I treat the agents as if they were junior developers capable of doing any dumb thing I ask them to, no matter how dumb it is. They, therefore, must be treated as junior devs - every line of code has to be reviewed. Every assumption about the specifications and requirements has to be checked against actual code, and against the original specifications and requirements.
What I see these days, is a lot of antsy kids who wanted to 100% ignore the wisdom of their elders, rushing into the maw of AI, and wondering why everyone is getting chewed up. Its pretty simple: AI-based software development is just another manifestation of software development, except that it requires even more rigorous quality steps in your workflow. So, if you were not rigorous before AI, you're going to get burned fingers - no doubt about it. Fix the rigor, people.
If you're not placing your AI buddy on a workflow that has "Specs->Reqs->Design->Analysis->Implementation->Review->Integration->Release" somewhere in the bag of worms, you're .. doing software wrong. You cannot just ignore natural laws and assume, because you 'know better', your software will 'be better'. And whether we like it or not, all software follows a philosophically natural law, which has evolved to become better understood, and thus more broadly applicable, over decades of human attention. Ignoring these natural laws in order to be a bleeding edge AI cowboy is only gonna get you butt-hurt, kiddo. Learn proper software management techniques first, AI second. Always. AI is just another junior dev - if your workflow is bogus, it doesn't matter how many dev's you've got. Period. You're going to be shipping crud.
It doesn't matter that AI-coding is taking over: if AI is being used in a brain-dead manner, then you should expect brain-dead results. You didn't review the code as the principle responsible party? The fault for the AI-induced failure nevertheless rests at your feet.
If, however, you apply decades of software development best-practices, you very definitely get living, vibrant, powerful results - the same as if you had a fleet of junior devs, assuming you treated them properly in the first place as well ..
Re: The user is visibly frustrated
#75Instead of reacting directly to the issue at hand I suggest you ponder what failure mode is being activated and why.
They are fundamentally not able to tell truth from fiction, but this also means they don't make errors like we do. They definitely create output we recognize as errors, but that's very different from our failure modes and you have to get used to it.
In my opinion it's better to branch off with an altered context that somehow avoids or mitigates the issue you're running into. Let's say they miss the mark. If you tell them "Don't do that" in the "conversation" this means the error is now and forever part of the context (assuming you stay within context limits and no compaction). Depending on their training this may or may not be detrimental to the quality of the rest of the conversation. You are now entering a section of their training where "error + someone swearing at them"-conversations have happened. I can't tell for sure, but my gut says this is not an advantageous place to be.
They are as I'm sure we all know completion engines and are in a very real way constantly cosplaying being productive "agents". They don't know if they are part of some type of modern Shakespearean play where sitting behind computers is part of the story or if they are in what we call "reality". By training on "conversations" they have become more likely to complete their input in a way that mimics what we call having a back and forth with some degree of technical accuracy.
In the extreme case you have a context that starts like "Please make all junior mistakes in this assignment. Make the code unreadable and be sure to include massive gotchas in subtle parts of the logic.". The results of this context won't be pretty. The other way around is not saying "Please make no errors", it's explaining in detail what you think is the right way. Coding style, if you care, architecture, etc. it all needs to be part of the context if you suspect it will substantially impact the completion. You have to imagine what real-life conversations have started with "Please make no errors". Again, I have no proof of course, but I have a strong feeling that human conversations that started with clearly and properly articulated specifications are qualitatively different from human conversations that started with "make no errors". In one you can see the pointy-haired boss and the other a seasoned engineer. Try to stay on the engineer side of their training.
I completely agree that they should be trained (or instructed) to react in a robotic tone stripped of all human pretense. We are trying to get at useful, general reasoning patterns latent in the data they trained on and, I regret to say, not the "human" parts which are usually a masterclass in cognitive biases and failures to reason.
Edit: the last sentence should be read in the voice of the Matrix's Architect.
Re: The user is visibly frustrated
#76We started using LLMs heavily at work this year and I switched from Vim to Zed to help with that. I now spend more time writing to the chat than editing code, and what I quickly learned to avoid frustration when I don't like the result was to git stash or reset the code and edit what I last wrote instead of trying to argue with the LLM. The chat doesn't have to be linear, it can branch off. Too bad we can't currently edit previous messages with Claude in Zed.
Repetitive issues are fixed by updating the memory or the prompt file, they can learn this way.
Also lately I noticed that Claude forget too much when compacting, so I just start a new session and it's easy when you spend a lot of time in plan mode to produce a written spec before implementation.
Re: The user is visibly frustrated
#77Working with LLMs is great for building communication skills. Communicating effectively is one of the hardest skills and it's baked into everything we do as humans. I'd say as a matter of principle: blame it on a communication failure on your end vs blaming the stupid LLM since you're the only one that can do anything about it. So I don't think it's a matter of form; whether the AI should or shouldn't act like a huma…
That's been one of the gravest re-realizations I've noticed watching coworkers trying to pick up "agentic" coding: they often just break down into "just fix it" or "why is this broke". I've noticed that even though supposedly there's training or some sort of work done to make the agent work better with unclear or ambiguous grammar or bad structure, it feels like the quality changes palpably when you talk in clear wel…
Agreed, 100%.
If you cannot formulate a specification, or describe a requirement - or indeed, if you cannot fathom the difference between a spec and a requirement, and why its needed to differentiate these from each other prior to doing a proper design and implementation - then you're going to carry your bad practice into the AI realm and that AI is going to be a force multiplier of your own bad practice. Because you will never know if the specs/reqs/design chosen by the AI are actually appropriate, unless you yourself review those specs/reqs/designs, AND the code produced by the AI to fulfill those specs/reqs/designs...
AI makes the able software developer, more able.
But it also makes the unable software developer, even more unable - with the risk of exceeding the AI-users limits on the Peter Principle scale.. in fact, AI will propel you to the middle of your own Peter Principle dilemma faster than you can type, probably.
Communication and writing skills are essential, with or without AI. But reading skills are even more relevant when dealing with AI. Alas, so few people who choose to use AI, have the temerity to actually do the work - or else they wouldn't be rushing for the AI tool in the first place.
Review, review, review. Always. Read the damn code, no matter who or what wrote it. Make sure it fulfills the specs and requirements its supposed to fulfill - and even more important make sure you, the reviewer, also understand the specs and requirements.
And if you don't, fix that - don't ship it anyway, ffs!
Re: The user is visibly frustrated
#78Instead of making tools we're making services. This is not confined to AI, it's everywhere. A tool does not fully solve your problem, it only goes in small steps. Yet these steps are predictable and consistent. A service attempts to solve your problem in a single step, yet the solution is only good if you match a predefined pattern. If you don't, then the service is of no use; there are no small steps you can combine to get where you need to.
Tools are very pleasant to use.
Re: The user is visibly frustrated
#79Working with LLMs is great for building communication skills. Communicating effectively is one of the hardest skills and it's baked into everything we do as humans. I'd say as a matter of principle: blame it on a communication failure on your end vs blaming the stupid LLM since you're the only one that can do anything about it. So I don't think it's a matter of form; whether the AI should or shouldn't act like a huma…
That's been one of the gravest re-realizations I've noticed watching coworkers trying to pick up "agentic" coding: they often just break down into "just fix it" or "why is this broke". I've noticed that even though supposedly there's training or some sort of work done to make the agent work better with unclear or ambiguous grammar or bad structure, it feels like the quality changes palpably when you talk in clear wel…
Yes, I have definitely witnessed this as well.
I think, I hope, this will be fixable to some degree, but at this moment I believe it's best to communicate in Queen's English and try to maintain the level of clarity of thought you expect of them in return.
My pet theory is that actual real conversations they were trained on with bad grammar and spelling are in general relatively starved of proper reasoning. By talking to them in this fashion you activate their lowbrow patterns and while it may not be catastrophic I can't imagine it helps.
Re: The user is visibly frustrated
#80The best advice I saw on this was to think of the LLM as simply a tool, and if you get bad results, it's because you -- the user of the tool -- are using it wrong.
After that, I'm less angry at the AI, and turn more towards a constructive "ok, this machine is stuck, how can I unstick" it approach. Calms the frustration a lot too.