Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why agents are bad pair programmers

justin.searls.co

Post reply on HN
Print view View original

101–110 of 239 posts

Re: Why agents are bad pair programmers

#101

Post by falcor84 »

psadri wrote: post #55
dragonfax wrote: post #37

Earlier quoted context omitted.

Claude Code has "plan mode" for this now. It enforces this behavior. But its still poorly documented.

They should add a “cmd-enter” for ask, and “enter” to go. Separately, if I were at cursor (or any other company for that matter), I’d have the AI scouring HN comments for “I wish x did y” suggestions.

I've been thinking about this a lot recently - having AI automate product manager user research. My thread of thought goes something like this:

0. AI can scour the web for user comments/complaints about our product and automatically synthesize those into insights.

1. AI research can be integrated directly into our product, allowing the user to complain to it just-in-time, whereby the AI would ask for clarification, analyze the user needs, and autonomously create/update an idea ticket on behalf of the user.

2. An AI integrated into the product could actually change the product UI/UX on its own in some cases, perform ad-hoc user research, asking the user "would it be better if things were like this?" and also measuring objective usability metrics (e.g. task completion time), and then use that validated insight to automatically spawn a PR for an A/B experiment.

3. Wait a minute - if the AI can change the interface on its own - do we even need to have a single interface for everyone? Perhaps future software would only expose an API and a collection of customizable UI widgets (perhaps coupled with official example interfaces), which each user's "user agent AI" would then continuously adapt to that user's needs?

Re: Why agents are bad pair programmers

#102

Post by throwawayffffas »

In my experience the problem is not they are too fast, they are too slow.

Honestly, their speed is just the right amount to make them bad. If they were faster, I could focus on following the code they are writing. But they take so much time for every edit that I tune out. On the other hand if they were slower, I could do other work while they are working, but they are done every 50 seconds to a few minutes which means I can't focus on other tasks.

If they did smaller faster changes it would probably be better.

Ideally though I would prefer them to be more autonomous, and the collaboration mode to be more like going over merge requests than pair programming. I ideally would like to have them take a task and go away for a few hours or even like 30 minutes.

The current loop, provide a task, wait 1 to 3 minutes, see a bunch of changes, provide guidance, repeat is the worst case scenario in my view.

Re: Why agents are bad pair programmers

#103

Post by jagged-chisel »

dkersten wrote: post #88
icedchai wrote: post #47

Earlier quoted context omitted.

Have you tried recently? This hasn't been my experience. I modify the code it's written, then ask it to reread the file. It generally responds "I see you changed file and [something.]" Or when it makes a change, I tell it I need to run some tests. I provide feedback, explain the problem, and it iterates. This is with Zed and Claude Sonnet.

I do notice though that if I edit what it wrote before accepting it, and then it sees it (either because I didn’t wait for it to finish or because I send it another message), it will overwrite my changes with what it had before my changes every single time, without fail. (Zed with Claude 4)

Gemini has insisted on remembering an earlier version of a file even after its own edits.

“We removed that, remember?”

“Yes! I see now …”

Sometimes it circles back to that same detail that no longer exists.

Re: Why agents are bad pair programmers

#104

Post by rhizome31 »

Macha wrote: post #99
rhizome31 wrote: post #91

As a developer who doesn't use AI for coding, except for the occasional non-project specific question to a chat bot, I am wondering if you use it for client projects or only for your own projects. If you do use it for client projects, do you have some kind of agreement that you're going to share their code with a third-party? I'm asking because most clients will make you sign a contract saying that you shouldn't disc…

I basically only use it in the workplace, and largely because of one of those AI mandates. I don't think it actually saves me enough time (or for many tasks, any time) so I wouldn't pay for it for my own projects, and also for my own projects, the enjoyability is a big factor, and I enjoy doing more than prompting.

Thank you for the reply. What do you mean by "AI mandates"? Does it mean your company has an explicit policy allowing sharing code with AI services?

Re: Why agents are bad pair programmers

#105

Post by rhizome31 »

internet_points wrote: post #92
rhizome31 wrote: post #91

As a developer who doesn't use AI for coding, except for the occasional non-project specific question to a chat bot, I am wondering if you use it for client projects or only for your own projects. If you do use it for client projects, do you have some kind of agreement that you're going to share their code with a third-party? I'm asking because most clients will make you sign a contract saying that you shouldn't disc…

I don't share anything with openai/anthropic that I wouldn't feel comfortable pasting into a web search prompt.

So no AI autocomplete I suppose?

I assume AI autocomplete may send any part of your code base or even all of it to a third-party.

Re: Why agents are bad pair programmers

#106

Post by Macha »

rhizome31 wrote: post #104
Macha wrote: post #99

Earlier quoted context omitted.

I basically only use it in the workplace, and largely because of one of those AI mandates. I don't think it actually saves me enough time (or for many tasks, any time) so I wouldn't pay for it for my own projects, and also for my own projects, the enjoyability is a big factor, and I enjoy doing more than prompting.

Thank you for the reply. What do you mean by "AI mandates"? Does it mean your company has an explicit policy allowing sharing code with AI services?

Sadly, I mean my current employer is doing the whole "tracking to see AI usage rates" and basically checking in performance reviews if people are using as much AI as the AI sales people told the CEO people need to use.

We're a SaaS company so we own all our code.

Re: Why agents are bad pair programmers

#107

Post by shultays »

A week or so ago I needed to convince chatgpt that following code will indeed initialize x values in struct

  struct MyStruct
  {
    int x = 5;
  };
  ...
  MyStruct myStructs[100];
It was insisting very passionately that you need MyStruct myStructs[100] = {}; instead.

I even showed msvc assembly output and pointed to the place where it is looping & assigning all x values and then it started hallucinating about msvc not conforming the standards. Then I did it for gcc and it said the same. It was surreal how strongly it believed it was correct.

Re: Why agents are bad pair programmers

#108

Post by UltraSane »

Bob_LaBLahh wrote: post #85
UltraSane wrote: post #43

It is rather soul crushing how fast LLMs spit out decent code.

In my experience, LLMs are idiot savant coders--but currently more idiot than savant. Claude 3.7 (via cursor and roo) can comment code well, create a starter project 10x faster than I could, and they spit out common crud apps pretty well. However I've come to the conclusion that LLMs are terrible at decision making. I would much rather have an intern architect my code than let AI do it. It's just too unreliable. It s…

LLMs currently can generate a few thousand lines of coherent code but they cannot write a cohesive large scale code base.

But LLMs are very good at writing SQL and Cypher queries that I would spend hours or days figuring out how to write.

Re: Why agents are bad pair programmers

#109

Post by rhizome31 »

Macha wrote: post #106
rhizome31 wrote: post #104

Earlier quoted context omitted.

Thank you for the reply. What do you mean by "AI mandates"? Does it mean your company has an explicit policy allowing sharing code with AI services?

Sadly, I mean my current employer is doing the whole "tracking to see AI usage rates" and basically checking in performance reviews if people are using as much AI as the AI sales people told the CEO people need to use. We're a SaaS company so we own all our code.

Wow, really?! I had no idea that such policies existed. Quite astonishing I have to say.

Re: Why agents are bad pair programmers

#110

Post by lodovic »

psadri wrote: post #36
khendron wrote: post #29

When I first tried an LLM agent, I was hoping for an interactive, 2-way, pair collaboration. Instead, what I got was a pairing partner who wanted to do everything themselves. I couldn't even tweak the code they had written, because it would mess up their context. I want a pairing partner where I can write a little, they write a little, I write a little, they write a little. You know, an actual collaboration.

I usually add “discuss first. Don’t modify code yet”. Then we do some back and forth. And finally, “apply”.

I try to be super careful, type the prompt I want to execute in a textfile. Ask the agent to validate and improve on it, and ask it to add an implementation plan. I even let another agent review the final plan. But even then, occasionally it still starts implementing halfway a refining.
Post reply on HN