Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Over-editing refers to a model modifying code beyond what is necessary

nrehiew.github.io

Post reply on HN
Print view View original

81–90 of 267 posts

Re: Over-editing refers to a model modifying code beyond what is necessary

#81

Post by esafak »

ramesh31 wrote: post #11
eterm wrote: post #3

It's funny, because the wisdom that was often taught ( but essentially never practiced ) was "Refactor as you go". The idea being that if you're working in an area, you should refactor and tidy it up and clean up "tech debt" while there. In practice, it was seldom done, and here we have LLMs actually doing it, and we're realising the drawbacks.

>The idea being that if you're working in an area, you should refactor and tidy it up and clean up "tech debt" while there. This is horrible practice, and very typical junior behavior that needs to be corrected against. Unless you wrote it, Chesterton's Fence applies; you need to think deeply for a long time about why that code exists as it does, and that's not part of your current task. Nothing worse than dealing wi…

Just do it in a follow up PR to keep them atomic. But do clean up; it's so easy now.

Re: Over-editing refers to a model modifying code beyond what is necessary

#83

Post by Bengalilol »

Tangent and admittedly off-topic but I've come to see LLM-assisted coding as a kind of teleportation.

With LLMs, you glimpse a distant mountain. In the next instant, you're standing on its summit. Blink, and you are halfway down a ridge you never climbed. A moment later, you're flung onto another peak with no trail behind you, no sense of direction, no memory of the ascent. The landscape keeps shifting beneath your feet, but you never quite see the panorama. Before you know it, you're back near the base, disoriented, as if the journey never happened. But confident, you say you were on the top of the mountain.

Manual coding feels entirely different. You spot the mountain, you study its slopes, trace a route, pack your gear. You begin the climb. Each step is earned steadily and deliberately. You feel the strain, adjust your path, learn the terrain. And when you finally reach the summit, the view unfolds with meaning. You know exactly where you are, because you've crossed every meter to get there. The satisfaction isn't just in arriving, nor in saying you were there: it is in having truly climbed.

Re: Over-editing refers to a model modifying code beyond what is necessary

#84

Post by whimblepop »

eterm wrote: post #3

It's funny, because the wisdom that was often taught ( but essentially never practiced ) was "Refactor as you go". The idea being that if you're working in an area, you should refactor and tidy it up and clean up "tech debt" while there. In practice, it was seldom done, and here we have LLMs actually doing it, and we're realising the drawbacks.

> In practice, it was seldom done, and here we have LLMs actually doing it, and we're realising the drawbacks.

I spent some time dealing with this today. The real issue for me, though, was that the refactors the agent did were bad. I only wanted it to stop making those changes so I could give it more explicit changes on what to fix and how.

Re: Over-editing refers to a model modifying code beyond what is necessary

#85

Post by dns_snek »

raincole wrote: post #13
anonu wrote: post #5

Here, the author means the agent over-edits code. But agents also do "too much": as in they touch multiple files, run tests, do deployments, run smoke tests, etc... And all of this gets abstracted away. On one hand, its incredible. But on the other hand I have deep anxiety over this: 1. I have no real understanding of what is actually happening under the hood. The ease of just accepting a prompt to run some script th…

It never ceases to scare me how they just run python code I didn't write via: > python > ${code the agent wrote on the spot} > EOF I mean, yeah, in theory it's just as dangerous as running arbitrary shell commands, which the agent is already doing anyway, but still...

The good news is that some of these harnesses (like Codex) use sandboxing. The bad news is that they're too inflexible to be effective.

By default these shell commands don't have network access or write access outside the project directory which is good, but nowhere near customizable enough. Once you approve a command because it needs network access, its other restrictions are lifted too. It's all or nothing.

Re: Over-editing refers to a model modifying code beyond what is necessary

#87

Post by jdkoeck »

Bengalilol wrote: post #83

Tangent and admittedly off-topic but I've come to see LLM-assisted coding as a kind of teleportation. With LLMs, you glimpse a distant mountain. In the next instant, you're standing on its summit. Blink, and you are halfway down a ridge you never climbed. A moment later, you're flung onto another peak with no trail behind you, no sense of direction, no memory of the ascent. The landscape keeps shifting beneath your f…

The thing is, with manual coding, you spot a view in the distance, you trek your way for a few hours, and you realize when you get there that the view isn’t as great as you thought it was.

With LLM-assisted coding, you skip the trek and you instantly know that’s not it.

Re: Over-editing refers to a model modifying code beyond what is necessary

#88

Post by ModernMech »

arjie wrote: post #77
ok_dad wrote: post #10

Earlier quoted context omitted.

Why are you letting the LLM drive? Don't turn on auto-approve, approve every command the agent runs. Don't let it make design or architecture decisions, you choose how it is built and you TELL that clanker what's what! No joke, if you treat the AI like a tool then you'll get more mileage out of it. You won't get 10x gains, but you will still understand the code.

I have never found any utility in that. After all, you can still just review the diffs and ask it for explanation for sections instead.

> After all, you can still just review the diffs

The diff: +8000 -4000

Re: Over-editing refers to a model modifying code beyond what is necessary

#89

Post by ValentineC »

Isolated_Routes wrote: post #56

I think building something really well with AI takes a lot of work. You can certainly ask it to do things and it will comply, and produce something pretty good. But you don't know what you don't know, especially when it speaks to you authoritatively. So checking its work from many different angles and making sure it's precise can be a challenge. Will be interesting to see how all of this iterates over time.

> But you don't know what you don't know, especially when it speaks to you authoritatively. So checking its work from many different angles and making sure it's precise can be a challenge.

I've spent far more time pitting one AI context against another (reviewing each other's work) than I have using AI to build stuff these days.

The benefit is that since it mostly happens asynchronously, I'm free to do other stuff.

Re: Over-editing refers to a model modifying code beyond what is necessary

#90

Post by lelanthran »

cortesoft wrote: post #63
ManuelKiessling wrote: post #36

Earlier quoted context omitted.

(I‘m saying this as someone who uses AI for coding a lot and mostly love it) Yeah, but is that really the same? Compilers work deterministically — if it works once, it will work always. LLMs are a different story for now.

LLMs are deterministic, too. I know there is randomness in the choosing tokens, but that randomness is derived from a random seed that can be repeated.

If I understand your argument, you're saying that models can be deterministic, right?

Care to point to any that are set up to be deterministic?

Did you ever stop to think about why no one can get any use out of a model with temp set to zero?

Post reply on HN