With this paper by Microsoft and the infamous paper by Apple last year, it seems the tech giants that don't have their own models are getting a bit insecure.
LLMs corrupt your documents when you delegate
151–160 of 235 posts
Re: LLMs corrupt your documents when you delegate
#152I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…
It could also be that much like most large orgs now you've made LLMs your entire personality, so you don't see the inherent bias. Most LLM users who are not touching code are certainly not going to be using a harness. They're going to take all the documents, slam all those tokens into the context window, see they have only used 500k out of their 1M tokens and say "summarize".
Wouldn't they be more likely to give ChatGPT access to a Google Drive folder or some such? The tools the agent has for editing documents will be whatever the app they used implemented.
Re: LLMs corrupt your documents when you delegate
#153Doesnt this apply to humans as well? Thats why children play the game "Telephone" and watch as a message gets corrupted. The solution is to provide single source of truth.
Re: LLMs corrupt your documents when you delegate
#154Earlier quoted context omitted.
It is, in fact, pretty common for people to throw a document at a language model along with a “make it more gooder” prompt.
That was true maybe 7 months ago. This is no longer the case. Harnesses use all kinds of tooling to edit things now.
I think you’re living in a bubble if you think the average user of AI even knows what a harness is
The vast majority of people are literally going to chatGPT, pasting in their document and asking for edits.
Re: LLMs corrupt your documents when you delegate
#155Earlier quoted context omitted.
It is, in fact, pretty common for people to throw a document at a language model along with a “make it more gooder” prompt.
That was true maybe 7 months ago. This is no longer the case. Harnesses use all kinds of tooling to edit things now.
People paste entire documents into gemini and chat gpt’s text boxes on the web and assume it will all turn out great
edit: apparently got beaten to this
Re: LLMs corrupt your documents when you delegate
#156[flagged]
Re: LLMs corrupt your documents when you delegate
#157I'm making tools for fighting this kind of degradation: https://github.com/JigSpec/JigSpec
Re: LLMs corrupt your documents when you delegate
#158Earlier quoted context omitted.
If you’re using LLMs for agentic work it is absolutely essential that you have a robust set of tools for them to use and the correct instructions to prompt their use. The LLM will come up with stupid ways to do things, common sense doesn’t exist for AI.
My favorite is when Claude will build a completely new application to load and inspect a .dll file using reflection instead of just googling the library's interfaces.
It did this for during one of the recent outrage periods. It was unjarring deps left and right instead of googling for it. What an easy way for me to own the tokenmaxxing leaderboard I remember thinking
Re: LLMs corrupt your documents when you delegate
#159> Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.
That's why harnesses and prompting rituals using dozens of markdown down files do not work as advertised and is pretty much snake oil otherwise known as "agentic engineering".
Also, the agentic engineering is pretty much so called prompt engineering except that prompt is now spread across dozens of markdown files directories.
Re: LLMs corrupt your documents when you delegate
#160Earlier quoted context omitted.
That was true maybe 7 months ago. This is no longer the case. Harnesses use all kinds of tooling to edit things now.
I think you’re living in a bubble if you think the average user of AI even knows what a harness is The vast majority of people are literally going to chatGPT, pasting in their document and asking for edits.
This will change too man. Maybe I am in a bubble but with how fast things are changing, it won’t be too long before the bubble becomes reality.
Either way we should be doing experiments on the actual capabilities of AI not about the stupidest possible way to use AI because it helps validate your own negative bias against AI.
Additionally as software engineers using agentic AI… which HN basically is… this experiment is not at all relevant in the context of where it is posted. We ALL use agentic ai and we all have the agent use surgical tools for editing. Don’t you find it strange that despite the fact we all do this, HN is full of rabid engineers gobbling this paper up as validation despite complete lack of relevance?