Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why agents are bad pair programmers

justin.searls.co

Post reply on HN
Print view View original

141–150 of 239 posts

Re: Why agents are bad pair programmers

#141

Post by SkyBelow »

>Continue to practice pair-programming with your editor, but throttle down from the semi-autonomous "Agent" mode to the turn-based "Edit" or "Ask" modes.

This can be done while staying in agent mode. I never used edit mode and only use ask mode when my question has nothing to do with the project I have open. Any other time, I tell it to either make no changes at all as I'm only asking a question to research something, or to limit changes to a much smaller scope and style. It doesn't work perfectly, but it works well enough that it is worth the tradeoff given the extra capabilities agent mode seems to provide (this likely depends upon the specific AI/LLM system you are using, so given another tool I might not arrive at the same conclusion).

Re: Why agents are bad pair programmers

#142

Post by bgro »

Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

Your ai is generating 2000 line code chunks? Are you prompting it to create the entire Skyrim game for SNES? Then after taking long lunch, getting mad when you press run and you find out it made fallout with only melee weapons in a ps1 style?

Re: Why agents are bad pair programmers

#143

Post by cmrdporcupine »

Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

I don't have this experience with Claude, frankly. I do have to correct it, but I try to give very specific prompts with very specific instructions. IT does well with highly commented code.

Now, I have the best luck in my personal project codebase, which I know extremely intimately so can be very surgical with.

Work, which has far less comments and is full of very high level abstractions that I don't know as well.. it struggles with. We both do.

It's a fine pair programmer when one of the pairs knows the codebase extremely well. It's a bad companion elsewhere.

Re: Why agents are bad pair programmers

#144

Post by catlifeonmars »

maleldil wrote: post #133
catlifeonmars wrote: post #84

Earlier quoted context omitted.

> My hot take is that if GitHub copilot is your window into llms, you’re getting the motel experience. I’ve long suspected this; I lean heavily on tab completion from copilot to speed up my coding. Unsurprisingly, it fails to read my mind a large portion of the time. Thing is, mind reading tab completion is what I actually want in my tooling. It is easier for me to communicate via code rather than prose, and I find t…

Take a look at aider's watch mode. It seems like a bridge for code completion with more powerful models than Copilot. https://aider.chat/docs/usage/watch.html

Thank you! I will check it out

Re: Why agents are bad pair programmers

#145

Post by jstummbillig »

Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

People are thinking too much of humans and LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort). This is simply not the case when doing AI coding, and you need to adjust how you work because of that and play to the strengths of this setup, if you want to get something out of it and not frustrate yourself.

Here is how to do this: Have it generate something. That first 2000 lines of not so great first attempt code, don't even think about understanding all of that, or, worse, about correcting it.

Review it loosely. You are not dealing with a human! There is absolutely no need to be thorough or nice. You are not hurting any feelings. Go for 80/20 (or the best ratio you think you can get).

Then, think:

- Anything you missed to inform the AI about? Update your initial prompt

- Anything the AI simply does not do well or to your liking? Write general instructions (all of the IDEs have some way of doing that) that are very explicit about what you don't want to see again, and what you want to see instead.

Then revert everything the ai did, and have it go again from the start. You should approach something that's better.

Re: Why agents are bad pair programmers

#146

Post by 34679 »

Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

Just like with human engineers, you need to start with a planning session. This involves a back and forth discussion to hammer out the details before writing any code. I start off as vague as possible to see if the LLM recommends anything I hadn't thought of, then get more detailed as I go. When I'm satisfied, I have it create 2 documents, initialprompt.txt and TODO.md. The initial prompt file includes a summary of the project along with instructions to read the to do file and mark each step as complete after finishing it.

This ensures the LLM has a complete understanding of the overall goals, along with a step by step list of tasks to get there. It also allows me to quickly get the LLM back up to speed when I need to start a new conversation due to context limits.

Re: Why agents are bad pair programmers

#147

Post by slfnflctd »

Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

> "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here"

This is the gist of what I've always wanted from a programming mentor, instructor, or tutor.

It can be surprisingly hard to find. Knowing that current LLMs still struggle with it perhaps helps explain why.

Re: Why agents are bad pair programmers

#148

Post by Sharlin »

jstummbillig wrote: post #145
Traster wrote: post #120

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations o…

People are thinking too much of humans and LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort). This is simply not the case when doing AI coding, and you need to adjust how you work because of that and play to the strengths of this setup, if you want to get something out of it and not frustrate yourself. Here is how to do this: Have…

This approach is essentially the PR workflow preferred by the author. Why let an LLM make huge changes to your working copy just for you to revert them next, instead of just writing patches to be asynchronously reviewed? What you propose is no way of doing pair programming in particular, and seems to support the author’s argument.

Re: Why agents are bad pair programmers

#149

Post by GrantMoyer »

searls wrote: post #116
rsynnott wrote: post #94

Earlier quoted context omitted.

I’ve always seen it as primarily an _education_ tool; the purpose of pair programming isn’t that two people pair programming are more productive than two people working individually, they’re generally not. So pair programming with a magic robot seems rather futile; it’s not going to learn anything.

LLMs in their current incarnation will not, but there's nothing inherently preventing them from learning. Contexts are getting large enough that having a sidecar database living with each project or individual as a sort of corpus of "shit I learned pairing with Justin" is already completely achievable, if only a product company wanted to do that.

Claude Plays Pokemon is kind of an interesting case study for this. This sort of knowledgebase is implemented, but even state of the art models struggle to use it effectively. They seem to fixate on small snippets from the knowledge base without any ability to consider greater context.

Re: Why agents are bad pair programmers

#150

Post by dsmurrell »

throwawayffffas wrote: post #102

In my experience the problem is not they are too fast, they are too slow. Honestly, their speed is just the right amount to make them bad. If they were faster, I could focus on following the code they are writing. But they take so much time for every edit that I tune out. On the other hand if they were slower, I could do other work while they are working, but they are done every 50 seconds to a few minutes which mean…

> that I tune out

You need a 30L fishtank for your desk. Great for tuning out.

Post reply on HN