Yeah I've been saying this for a while: AI-washing any text will degrade it, compounding with each pass. "Semantic ablation" is my favorite term for it: https://www.theregister.com/software/2026/02/16/semantic-abl...
It is X, not Y!
LLMs corrupt your documents when you delegate
131–140 of 235 posts
Re: LLMs corrupt your documents when you delegate
#132Earlier quoted context omitted.
By „with each pass” do you mean within the same session, or with new session (context window) each time?
In my experience, it happens with each edit of the document, whether or not you clear the context window. You can somewhat mitigate this, at the same moment you ask for the new edit, by adding new info or specifying the lost meaning you want to add back. But other things will still get washed out. Nuances will drift, sharp corners will be ablated. You're doing a Xerox copy of your latest Xerox copy, so even if you ad…
Which is why I think AI assisted writing is better then just letting it write the full text (if you care about the quality of the result). The act of writing isn't just the production of text, it is about wrangling a topic, rotating it in your mind and finding the perfect expression for a thought you have and that you want to convey to others. Some of those things can't be known by the LLM since you don't know them yourself by the point you started out.
Often that thinking bit itself provides value to the person doing it, beyond the text itself. By letting a LLM do it for you, you rob yourself of the change of thought and the new findings you may encounter.
Working with LLMs just makes it quicker to get going, bit you need to be a ruthless editor.
Re: LLMs corrupt your documents when you delegate
#133Earlier quoted context omitted.
I always thought it was a little weird that LLMs aren't sophisticated enough to surgically edit files as needed. For example, if there is a code block that needs to be wrapped within another function call, it'll rewrite the entire function call and you'll just have to pray that the re-written code block wasn't subtly changed. I _think_ so far it hasn't introduced any changes....
You can just look at the diff when you do a pull request, no prayer needed, and if you want it to be “surgical” in that way, your prompt (and agents.md) can be specific. You can also unit test the function to better assure behavior didn’t change.
Indeed, that's what I do. I inspect the diff, though if it's an indentation change the entire block will be marked changed.
Still not an excuse to not read every line of course...
Unit tests give me the confidence that at least those tested logic paths are unaffected.
Sometimes with older codebases one cannot assume the paths have adequate test coverage.
Re: LLMs corrupt your documents when you delegate
#134Earlier quoted context omitted.
I’ve seen people drink their own recycled piss and inject coffee into their ass - what’s your point?
In the first half, I thought you were an astronaut, but the second half has me double-guessing myself.
I used to be a connoisseur of weird Facebook groups - I would advise everyone to never look into aged urine, coffee enemas or targeted individuals - makes you lose your faith in humanity
Re: LLMs corrupt your documents when you delegate
#135I think the problem is that we're using LLMs to do too much of the work. We should aim to design agents that use the LLM as the thinnest possible layer to translate the natural language intent into a deterministic process, minimizing round trips to the LLM as much as possible.
A process isn't automated until neither human nor genie is in the loop.
Re: LLMs corrupt your documents when you delegate
#136[flagged]
Re: LLMs corrupt your documents when you delegate
#137Earlier quoted context omitted.
If you’re using LLMs for agentic work it is absolutely essential that you have a robust set of tools for them to use and the correct instructions to prompt their use. The LLM will come up with stupid ways to do things, common sense doesn’t exist for AI.
My favorite is when Claude will build a completely new application to load and inspect a .dll file using reflection instead of just googling the library's interfaces.
“Use all of the tools at your disposal, including searching the internet” is my claude-specific common instruction.
Re: LLMs corrupt your documents when you delegate
#138[flagged]
Re: LLMs corrupt your documents when you delegate
#139I think the problem is that we're using LLMs to do too much of the work. We should aim to design agents that use the LLM as the thinnest possible layer to translate the natural language intent into a deterministic process, minimizing round trips to the LLM as much as possible.
This becomes clear to anyone that wants to do marginally complex work. Developing pipelines that combine pre-processing flows, semantic targeting, and minimal contextual calls to an LLM API gets you powerful automated steps. Combined with separate validation steps, LLMs go from toys to useful.
Re: LLMs corrupt your documents when you delegate
#140Earlier quoted context omitted.
I agree with most of what you wrote except for this: >Frequent LLM users already know not to do that. And I think that’s the biggest problem. Amidst the current push to utilize LLMs across orgs and groups there are a large (if even say majority) of people that are using them every day but who have never approached anything as technical as a “harness” before let alone an entire setup. For them the behavior mentioned h…
Exactly - I am a lawyer and we are told to use dedicated AI products as much and however we want. There will be errors made
Much to the often-reported chagrin of judges across the country.