LLMs corrupt your documents when you delegate • Hacker News Board

Re: LLMs corrupt your documents when you delegate

#21

Post by dangus » Sat, May 09, 2026, 2:28 PM UTC

adampunk wrote: post #18

dangus wrote: post #15

Earlier quoted context omitted.

A human can sit down and say “I’m going to make sure this is correct on the first pass and make sure I make an exact copy.” They have cognitive awareness of which tasks are highly critical and need more checking and re-checking without being prompted to think that way. For a human, time doesn’t stop when the first pass of the prompt and response is over. An LLM effectively wipes its memory of what it just did unless…

[flagged]

And that’s why I’m paid six figures and my LLM is paid $20/month.

Re: LLMs corrupt your documents when you delegate

#22

Post by simonh » Sat, May 09, 2026, 2:33 PM UTC

adampunk wrote: post #18

dangus wrote: post #15

Earlier quoted context omitted.

A human can sit down and say “I’m going to make sure this is correct on the first pass and make sure I make an exact copy.” They have cognitive awareness of which tasks are highly critical and need more checking and re-checking without being prompted to think that way. For a human, time doesn’t stop when the first pass of the prompt and response is over. An LLM effectively wipes its memory of what it just did unless…

[flagged]

Humans understand what mistakes are and can reason about what constitutes a mistake and what doesn’t. LLMs can’t do that.

It’s for the same reason that they will invent bullshit instead of saying “I don’t know”, when they don’t know. They don’t have a concept of accuracy of facts.

Re: LLMs corrupt your documents when you delegate

#23

Post by meander_water » Sat, May 09, 2026, 2:36 PM UTC

> We find that models are not failing due to “death by a thousand cuts” (i.e., many small errors). Instead, they main- tain near-perfect reconstruction in some rounds, and experience critical failures in a few rounds, typically losing 10-30+ points in a single round trip

> We find that weaker models’ degradation originates primarily from content deletion, while frontier models’ degradation is attributable to corruption of content.

I think we largely already knew this. This is why we fudge around with harnesses and temperature etc.

Re: LLMs corrupt your documents when you delegate

#24

Post by simonw » Sat, May 09, 2026, 2:37 PM UTC

adampunk wrote: post #8

LLMs will make mistakes on every turn. The mistakes will have little to no apparent connection to "difficulty" or what may or may not be prevalent in the training data. They will be mistakes at all levels of operation, from planning to code writing to reporting. Whether those mistakes matter and whether you catch them is mostly up to you. I have yet to find a model that does not make mistakes each turn. I suspect tha…

> I have yet to find a model that does not make mistakes each turn

What kind of mistakes are you talking about here?

Re: LLMs corrupt your documents when you delegate

#25

Post by threethirtytwo » Sat, May 09, 2026, 2:38 PM UTC

This experiment needs to be put in perspective. Let me explain. IF you did this SAME experiment with a human and had a human read an ENTIRE document and then reproduce said document with edits. The DOCUMENT would DEGRADE even more.

The way this experiment is conducted is not inline with how current agentic AI is used OR how even humans edit documents.

Here's how agentic AI currently typically do edits:

1. They read the whole document. 2. They come up with a patch. A diff of the section they want to edit. 3. They change THAT section only.

This is NOT what that experiment was doing. A 25% degradation rate would render the whole industry dead. No one would be using claude code because of that. The reality is... everyone is using claude code.

AI is alien to the human brain, but in many ways it is remarkably. This is one aspect of similarity in that we cannot edit a whole document holistically to produce one edit. It has to be targeted surgical edits rather then a regurgitation of the entire document with said edit.

Re: LLMs corrupt your documents when you delegate

#26

Post by threethirtytwo » Sat, May 09, 2026, 2:44 PM UTC

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically.

The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have to ingest the document hundreds of times to achieve a state called "memorization". The equivalent in an LLM is called training. If you train a document into an LLM you can get parity with the memorized human edit in this case.

But the above is irrelevant. The point is LLMs have certain similarities with humans. You need to design a harness such that an LLM edits a document the same way a human would: Search and surgical edits. All coding agents edit this way, so this paper isn't relevant.

Re: LLMs corrupt your documents when you delegate

#27

Post by Twirrim » Sat, May 09, 2026, 2:46 PM UTC

timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

A coworker talks about LLMs as "bullshit" layers. Not exactly dismissing them or being derogatory about them, but emphasising that each time you feed something through an LLM, what comes out the other side may not be what you expect/want. Like that guy at the pub sharing what he'd seen online somewhere, after a few pints. Might be accurate, but carries notable risk it's not.

So e.g., don't use an LLM to call an API to gather data and produce a report on it, as that's feeding deterministic data through a "bullshit" layer, meaning you can't trust what comes out the other side. Instead use the LLM to help you write the code that will produce a deterministic output from deterministic data.

I've seen co-workers use LLMs to summarise deterministic data coming from APIs and have reports be wildly off the mark as often as they are accurate. Depending on what they're looking at that can have catastrophic risk.

Re: LLMs corrupt your documents when you delegate

#28

Post by threethirtytwo » Sat, May 09, 2026, 2:51 PM UTC

timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

A human doing the same tasks as what the LLM did in the paper that the human will degrade the document further then the LLM. If the LLM is 25%, a human would degrade it probably 80% if they used the same technique as the LLM did in this paper. I'm talking about a single pass.

The fact of the matter is, humans don't edit things the way it was done in the paper and neither do coding agents like claude. Think about it: You do not ingest an entire paper and then regurgitate that paper with a single targeted edit... and neither do coding agents.

Also think carefully. A 25% degradation rate is unacceptable in the industry. The AI change that's taking over all of SWE development would not actually exist if there was 25% degradation... that's way too much.

Re: LLMs corrupt your documents when you delegate

#29

Post by arian_ » Sat, May 09, 2026, 2:54 PM UTC

[flagged]

Re: LLMs corrupt your documents when you delegate

#30

Post by ekidd » Sat, May 09, 2026, 2:56 PM UTC

timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

Where this result is actually interesting and relevant is when a coding agent splits a large source file into multiple smaller files. Opus + Claude Code will try to recite long sections of source code from memory into each of the new files, instead of using some sort of copy/paste operation like a human would.

Moving a file is a bit easier. LLMs may sometimes try to recite the file from memory. But if you tell them to use "git mv" and fix the compiler errors, they mostly will.

Ordinary editing on the other hand, generally works fine with any reasonable model and tool setup. Even Qwen3.6 27B is fine at this. And for in-place edits, you can review "git diff" for surprises.