Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Over-editing refers to a model modifying code beyond what is necessary

nrehiew.github.io

Post reply on HN
Print view View original

101–110 of 267 posts

Re: Over-editing refers to a model modifying code beyond what is necessary

#101

Post by standardly »

I've had a bad experience using AI for front-end stuff, where I replace or deprecate a feature only to notice later all the artifacts it left behind, some which were never even used in the first place.

I re-did an entire UI recently, and when one of the elements failed to render I noticed the old UI peeking out from underneath. It had tried just covering up old elements instead of adjusting or replacing them. Like telling your son to clean their room, so they push all the clothes under the bed and hope you don't notice LOL

It saves 2 hours of manual syntax wrangling but introduces 1 .5 hours of clean up and sanity checking. Still a net productivity increase, but not sure if its worth how lazy it seems to be making me (this is an easy error to correct, im sure, but meh Claude can fix it in 2 seconds so...)

Re: Over-editing refers to a model modifying code beyond what is necessary

#102

Post by soiltype »

JoshTriplett wrote: post #57
soiltype wrote: post #45

Earlier quoted context omitted.

Personally I've found "carefully review every move it makes" to be an extremely unpleasant and difficult workflow. The effort needed to parse every action is immense, but there's a complete absence of creative engagement - no chance of flow state. Just the worst kind of work which I've been unable to sustain, unfortunately. At this point I mostly still do work by hand.

I agree, but I also think that giving the LLM free rein is also extremely unpleasant and difficult. And you still need to review the resulting code.

I don't think there's anything difficult or unpleasant about the process of letting the LLM run free, that's the whole point, it's nearly frictionless. Which includes not reviewing the code carefully. You say "need" but you mean "ought".

Re: Over-editing refers to a model modifying code beyond what is necessary

#103

Post by qurren »

btbuildem wrote: post #91

I wish there was a reliable way to choke the agents back and prevent them from doing this. Every line of code added is a potential bug, and they overzealously spew pages and pages of code. I've routinely gone through my (hobby) projects and (yes, still with the aid of an LLM) trimmed some 80% of the generated code with barely any loss of functionality. The cynic in me thinks it's done on purpose to burn more tokens.…

> if we had access to all the knobs and levers.

We do, just tell it what you want in your AGENTS.md file.

Agents also often respond well to user frustration signs, like threatening to not continue your subscription.

Re: Over-editing refers to a model modifying code beyond what is necessary

#104

Post by recursivecaveat »

My experience is usually the opposite. The code they write is verbose yes, but the diffs are over-minimal. Whenever I see a comment like "Tool X doesn't support Y or has a bug with Z [insert terrible kludge]" and actually fixing the problem in the other file would be very easy, I know it is AI-generated. I suspect there is a bias towards local fixes to reduce token usage.

Re: Over-editing refers to a model modifying code beyond what is necessary

#105

Post by m463 »

You know, this made me think of over-engineering.

...and that led me to believe that AI might be very capable to develop over-engineered audio equipment. Think of all the bells and whistles that could be added, that could be expressed in ridiculous ways with ridiculous price tags.

Re: Over-editing refers to a model modifying code beyond what is necessary

#106

Post by janalsncm »

This is a really solid writeup. LLMs are way too verbose in prose and code, and my suspicion is this is driven mainly by the training mechanism.

Cross entropy loss steers towards garden path sentences. Using a paragraph to say something any person could say with a sentence, or even a few precise words. Long sentences are the low perplexity (low statistical “surprise”) path.

Re: Over-editing refers to a model modifying code beyond what is necessary

#107

Post by mncharity »

_pastel wrote: post #53
jstanley wrote: post #4

Conversely, I often find coding agents privileging the existing code when they could do a much better job if they changed it to suit the new requirement. I guess it comes down to how ossified you want your existing code to be. If it's a big production application that's been running for decades then you probably want the minimum possible change. If you're just experimenting with stuff and the project didn't exist at…

The tradeoff is highly contextual; it's not a tradeoff an agent can always make by inspecting the project themselves. Even within the same project, for a given PR, there are some parts of the codebase I want to modify freely and some that I want fixed to reduce the diff and testing scope. I try to explain up-front to the agent how aggressively they can modify the existing code and which parts, but I've had mixed succ…

Just brainstorming, but perhaps a more tangible gradient, with social backpressure? Imagine three identical patch tools: "patch", "submit patch", and "send patch to chief architect and wait". With the "where each can be used" explained or even enforced. Having the contrast of less-aggressive options, might make it easier to encourage more aggressive refactoring elsewhere. Or pushing the impact further up the CoT, "patch'ing X requires an analysis field describing less invasive alternatives and their un/suitability; for Y, just do it, refactor aggressively".

Re: Over-editing refers to a model modifying code beyond what is necessary

#108

Post by lambda »

soiltype wrote: post #45
ok_dad wrote: post #10

Earlier quoted context omitted.

Why are you letting the LLM drive? Don't turn on auto-approve, approve every command the agent runs. Don't let it make design or architecture decisions, you choose how it is built and you TELL that clanker what's what! No joke, if you treat the AI like a tool then you'll get more mileage out of it. You won't get 10x gains, but you will still understand the code.

Personally I've found "carefully review every move it makes" to be an extremely unpleasant and difficult workflow. The effort needed to parse every action is immense, but there's a complete absence of creative engagement - no chance of flow state. Just the worst kind of work which I've been unable to sustain, unfortunately. At this point I mostly still do work by hand.

This is my biggest problem with the promises of agentic coding (well, there are an awful lot of problems, but this is the biggest one from an immediate practical perspective).

One the one hand, reviewing and micromaning everything it does is tedious and unrewarding. Unlike reviewing a colleague's code, you're never going to teach it anything; maybe you'll get some skills out of it if you finds something that comes up often enough it's worth writing a skill for. And this only gets you, at best, a slight speedup over writing it yourself, as you have to stay engaged and think about everything that's going on.

Or you can just let it grind away agentically and only test the final output. This allows you to get those huge gains at first, but it can easily just start accumulating more and more cruft and bad design decisions and hacks on top of hacks. And you increasingly don't know what it's doing or why, you're losing the skill of even being able to because you're not exercising it.

You're just building yourself a huge pile of technical debt. You might delete your prod database without realizing it. You might end up with an auth system that doesn't actually check the auth and so someone can just set a username of an admin in a cookie to log in. Or whatever; you have no idea, and even if the model gets it right 95% of the time, do you want to be periodically rolling a d20 and if you get a 1 you lose everything?

Re: Over-editing refers to a model modifying code beyond what is necessary

#109

Post by qweiopqweiop »

simonw wrote: post #58

I've not seen over-editing in Claude Code or Codex in quite a while, so I was interested to see the prompts being used for this study. I think they're in here, last edited 8 months ago: https://github.com/nreHieW/fyp/blob/5a4023e4d1f287ac73a616b5...

Likewise, this felt like an early agent problem to me.

Re: Over-editing refers to a model modifying code beyond what is necessary

#110

Post by stared »

anonu wrote: post #5

Here, the author means the agent over-edits code. But agents also do "too much": as in they touch multiple files, run tests, do deployments, run smoke tests, etc... And all of this gets abstracted away. On one hand, its incredible. But on the other hand I have deep anxiety over this: 1. I have no real understanding of what is actually happening under the hood. The ease of just accepting a prompt to run some script th…

I really recommend using Agent Safehouse (https://news.ycombinator.com/item?id=47301085).

Don’t give your agent access to content it should not edit, don’t give keys it shouldn’t use.

Post reply on HN