Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

111–120 of 235 posts

Re: LLMs corrupt your documents when you delegate

#111

Post by CamperBob2 »

8note wrote: post #106
CamperBob2 wrote: post #89

Earlier quoted context omitted.

So you'll be able to show me the early-90s era program that can solve original IMO-level problems when supplied with the plaintext questions. Right?

if i presented math problems to the best english mathematicians in chinese, does that mean they arent able to reason? the plain text is an arbitrary constraint

The actual question is, if you presented an undergraduate-level calculus problem to a human who is considered intelligent but who was never given an "understanding" of math in school, would the human be able to solve it? Why or why not?

If so, what exactly would you call the process by which the intelligent human solves the math problem that he or she does not initially understand?

Whatever you call that process is what a reasoning model does. You don't have to call it "reasoning," of course... unless you want other people to understand what you're talking about.

Re: LLMs corrupt your documents when you delegate

#112

Post by daveguy »

woeirua wrote: post #7

It's an interesting paper, but I'd like to see a lot more about the types of errors that the LLM makes. Are they happening in the forward pass or the inverse pass? My guess is the inverse pass.

This sounds like wishful thinking to me.

The tasks are designed to be reversible. Whether it stochastic parrots in the forward direction or reverse direction is irrelevant. Especially considering these are inference engines. Every pass is a forward pass from the perspective of the LLM / agent. There is no feedback loop, and part of the reason why it's so easy for these things to mangle tasks. They are plausible sounding sentence/sequence generators.

Re: LLMs corrupt your documents when you delegate

#113

Post by redsocksfan45 »

shahbaby wrote: post #104
threethirtytwo wrote: post #26

Earlier quoted context omitted.

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically. The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have…

> People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. OR it could be because their concerns are genuine but are ignored in favour of a good sounding story.

[dead]

Re: LLMs corrupt your documents when you delegate

#114

Post by ofjcihen »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

I agree with most of what you wrote except for this:

>Frequent LLM users already know not to do that.

And I think that’s the biggest problem. Amidst the current push to utilize LLMs across orgs and groups there are a large (if even say majority) of people that are using them every day but who have never approached anything as technical as a “harness” before let alone an entire setup.

For them the behavior mentioned here is a major issue.

Re: LLMs corrupt your documents when you delegate

#115

Post by rs545837 »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

[flagged]

Re: LLMs corrupt your documents when you delegate

#116

Post by ofjcihen »

deadbabe wrote: post #65
embedding-shape wrote: post #54

Earlier quoted context omitted.

I don't think there is a single reason. Models are improving, so are the harnesses, prompts and we who use them a lot also get more proficient and learn where they can be used effectively vs not, so lots of improvements all over the ecosystem, brought together. Latest big change is probably how feasible local models are becoming, like Qwen 3.6 and Gemma 4, they're no longer easily getting stuck in loops and repetitio…

> we who use them a lot also get more proficient and learn where they can be used effectively vs not I think it’s always been obvious where an LLM could be used effectively and where it cannot, if you understand how they work and don’t see them as magical. The “increase in proficiency” is mostly people coming back to reality and being more intentional about LLM usage. There are no surprise discoveries here. One does…

I think you hit the nail on the head. I had been in this space for a little bit before it really became popular. I haven’t seen incredible gains in model competency. What I have seen though is people figuring out what works and what doesn’t.

Re: LLMs corrupt your documents when you delegate

#117

Post by kristjansson »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

Only sort of related, but I would love to see a harness with ed as the primary file editing / reading tool. Half the bash Claude runs seems to be sed anyway, having some state persist in ed would seem to help.

What does one do when a full editor consumes too much bandwidth^H tokens? Use ed, the standard editor!

Re: LLMs corrupt your documents when you delegate

#118

Post by ACCount37 »

tommyage wrote: post #82
ACCount37 wrote: post #40

Earlier quoted context omitted.

No. The whole point of creating software is getting things done . "More accurately and consistently" was merely downstream from what capabilities were natural for machine logic and hard algorithms. Now, we're just spoiled for choice. We have hard algorithm software where we want to do things that benefit for accurate, consistent, highly deterministic behavior - and we have soft algorithm AI for when we want to do thi…

All of software is hard-coded algorithm. If you differ between AI source code and engineer source code say so. "Getting things done" is a business need. Which things get translated to a deterministic language executable by a computer is code. There are entire languages dedicated for lesser engineers/domain experts to formulate business requirements. Anyhow; What's your point? That we received a framework for "soft al…

AI is more "grown" than it is "hard-coded". It's sideways to normal software - the way DSP is sideways to normal software but somehow even worse.

It all comes down to hard logic eventually, but that "eventually" has teeth. None of the interesting behaviors of AI systems live in "engine.py".

My point is: there are tasks where the choices are to use AI, use a meatbag, or suck forever. The "use AI" option going to be flawed, and often in the same ways "use meatbag" is. But it's going to be cheaper, much more scalable, and a lot better than "suck forever". Humanlike flaws are the price you pay for accessing humanlike capabilities.

Re: LLMs corrupt your documents when you delegate

#119

Post by tieTYT »

threethirtytwo wrote: post #26
simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically. The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have…

> a human will DO worse then a 25% degradation

As I was reading this article, a similar thought occurred to me: "I wonder if that's better or worse than a human?" Unfortunately, there was no human baseline in this study. That said, there are studies that compare LLM to human performance. Usually, humans perform much better (like 5-7x better) at long-running tasks.

In other words, a human would probably do better than an LLM on this task.

Humans lose to LLMs in narrow, well-specified text/symbolic reasoning tasks where the model can exploit breadth, speed, and search. Usually, the LLM performed ~15% better than humans, but I saw studies that were as high as 80%. To my surprise, these studies were usually about "soft skills" like creativity and persuasion.

Re: LLMs corrupt your documents when you delegate

#120

Post by javajive »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

[dead]
Post reply on HN