Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Over-editing refers to a model modifying code beyond what is necessary

nrehiew.github.io

Post reply on HN
Print view View original

171–180 of 267 posts

Re: Over-editing refers to a model modifying code beyond what is necessary

#171

Post by cortesoft »

Supermancho wrote: post #124
cortesoft wrote: post #61

Earlier quoted context omitted.

At the end of the day, LLMs are also deterministic. They are running on computers just like all software, and if you have all the same data and random seeds, and you give the same prompt to the same LLM, you will get back the exact same response.

> you give the same prompt to the same LLM, you will get back the exact same response. Demonstrably incorrect. This is because the model selection, among other data, is not fixed for (I would say most) LLMs. They are constantly changing. I think you meant something more like an LLM with a fixed configuration. Maybe additional constraints, depending on the specific implementation.

Yes, by 'same LLM', I mean literally the same model with the same random seeds. You are correct, the big LLMs from providers like Anthropic and OpenAI do not meet this definition.

Re: Over-editing refers to a model modifying code beyond what is necessary

#172

Post by arjie »

ModernMech wrote: post #88
arjie wrote: post #77

Earlier quoted context omitted.

I have never found any utility in that. After all, you can still just review the diffs and ask it for explanation for sections instead.

> After all, you can still just review the diffs The diff: +8000 -4000

You can ask it to make the changes in appropriate PRs. SOTA model + harness can do it. I find it useful to separate refactors and implementations, just like with humans, but I admittedly rely heavily on multi-provider review.

Re: Over-editing refers to a model modifying code beyond what is necessary

#173

Post by rtpg »

hathawsh wrote: post #167
rtpg wrote: post #162

Earlier quoted context omitted.

Are you writing code that gets reviewed by other people? Were code reviews hard in the past? Do your coworkers care about "code quality" (I mean this in scare quotes because that means different things to different people). Are you working more on operational stuff or on "long-running product" stuff? My personal headcanon: this tooling works well when built on simple patterns, and can handle complex work. This toolin…

You hinted at an aspect I probably haven't considered enough: The code I'm working on already has many well-established, clean patterns and nearly all of Claude's work builds on those patterns. I would probably have a very different experience otherwise.

I legit think this is the biggest danger with velocity-focused usage of these tools. Good patterns are easy to use and (importantly!) work! So the 32nd usage of a good pattern will likely be smooth.

The first (and maybe even second) usage of a gnarly, badly thought out pattern might work fine. But you're only a couple steps away from if statement soup. And in the world where your agent's life is built around "getting the tests to pass", you can quickly find it doing _very_ gnarly things to "fix" issues.

Re: Over-editing refers to a model modifying code beyond what is necessary

#174

Post by hexaga »

foo12bar wrote: post #80

I've noticed AI's often try and hide failure by catching exceptions and returning some dummy value maybe with some log message buried in tons of extraneous other log messages. And the logs themselves are often over abbreviated and missing key data to successfully debug what is happening. I suspect AI's learned to do this in order to game the system. Bailing out with an exception is an obvious failure and will be pena…

AI behavior is pretty easy to understand and predict if you view it from the lens of: they will shamelessly do any/everything possible to game whatever metric they are trained on. Because... that's how hill-climbing a metric looks. It's A/B enshittification taken to inscrutable heights.

They are trained on human feedback, so there is no other way this goes. Every bit of every response is pointed toward subversion of the assumed evaluator.

Re: Over-editing refers to a model modifying code beyond what is necessary

#175

Post by nunez »

Isolated_Routes wrote: post #56

I think building something really well with AI takes a lot of work. You can certainly ask it to do things and it will comply, and produce something pretty good. But you don't know what you don't know, especially when it speaks to you authoritatively. So checking its work from many different angles and making sure it's precise can be a challenge. Will be interesting to see how all of this iterates over time.

Yep. It's quite good at getting you to 80% of the solution. The other 20% depends on the problem!

Re: Over-editing refers to a model modifying code beyond what is necessary

#177

Post by nunez »

graybeardhacker wrote: post #37

I use Claude Code every day and have for as long as it has been available. I use git add -p to ensure I'm only adding what is needed. I review all code changes and make sure I understand every change. I prompt Claude to never change only whitespace. I ask it to be sure to make the minimal changes to fix a bug. Too many people are treating the tools as a complete replacement for a developer. When you are typing a text…

i use claude in more or less the same way but it sure is tempting to just glaze over the 300+ line diffs it produces.

Re: Over-editing refers to a model modifying code beyond what is necessary

#178

Post by rcvassallo83 »

This resonates

I've had success with greenfield code followed by frustration when asking for changes to that code due to over editing

And prompting for "minimal changes" does keep the edits down. In addition to this instruction, adding specifics about how to make the change and what not to do tends to get results I'm looking for.

"add one function that does X, add one property to the data structure, otherwise leave it as is, don't add any new validation"

Re: Over-editing refers to a model modifying code beyond what is necessary

#179

Post by anabis »

Swizec wrote: post #155
hathawsh wrote: post #142

I'm either in a minority or a silent majority. Claude Code surpasses all my expectations. When it makes a mistake like over-editing, I explain the mistake, it fixes it, and I ask it to record what it learned in the relevant project-specific skills. It rarely makes that mistake again. When the skill file gets big, I ask Claude to clean and compact it. It does a great job. It doesn't really make sense economically for…

> I'm either in a minority or a silent majority. Claude Code surpasses all my expectations. I looked at some stats yesterday and was surprised to learn Cursor AI now writes 97% of my code at work. Mostly through cloud agents (watching it work is too distracting for me) My approach is very simple: Just Talk To It People way overthink this stuff. It works pretty good. Sharing .md files and hyperfocusing on various orch…

Are you mostly using the Composer model?

Re: Over-editing refers to a model modifying code beyond what is necessary

#180

Post by Swizec »

anabis wrote: post #179
Swizec wrote: post #155

Earlier quoted context omitted.

> I'm either in a minority or a silent majority. Claude Code surpasses all my expectations. I looked at some stats yesterday and was surprised to learn Cursor AI now writes 97% of my code at work. Mostly through cloud agents (watching it work is too distracting for me) My approach is very simple: Just Talk To It People way overthink this stuff. It works pretty good. Sharing .md files and hyperfocusing on various orch…

Are you mostly using the Composer model?

> Are you mostly using the Composer model?

Don’t really think about it. I think when I talk to it through Slack, cursor users codex, in my ide looks like it’s whatever highest claude. In Github comments, who even knows

Post reply on HN