Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

41–50 of 235 posts

Re: LLMs corrupt your documents when you delegate

#41

Post by mpyne »

ben_w wrote: post #33
Twirrim wrote: post #27

Earlier quoted context omitted.

A coworker talks about LLMs as "bullshit" layers. Not exactly dismissing them or being derogatory about them, but emphasising that each time you feed something through an LLM, what comes out the other side may not be what you expect/want. Like that guy at the pub sharing what he'd seen online somewhere, after a few pints. Might be accurate, but carries notable risk it's not. So e.g., don't use an LLM to call an API t…

Similar experience. I wouldn't say it even needs to be like some random person in the local pub: this behaviour is what you'd get from any game of telephone, book authors will say how you need to be blunt and direct about points in the book because readers will miss subtlety, anyone who has been quoted in a newspaper will have a story about the paper getting it wrong, etc. However, there's a reason pre-computing bure…

> However, there's a reason pre-computing bureaucracy came with paper trails and meeting minutes getting written up, why court cases are increasingly cautious about the reliability of eye witnesses.

This was the comment I was coming in to make: I worked in a pre-computing bureaucracy (the U.S. Navy's) and "staff you delegated work to have consistent trouble following the directions you provide for the delegated work" is just a fact of life.

A lot of it is telephone game, a lot of it is is lack of real familiarity with office software, a lot of it is the inherent integration challenge from sending the same document out for coordination to dozens of stakeholders.

All those mistakes you made fixes for based on comments in the draft that went out for O-6 review? At least 2 will pop up again at 1-star review because staffers will copy the same text back out from their local copy they had stashed during O-6 review rather than re-reviewing from scratch.

Style guidance to meet the Admiral's preferred format? You can provide it but there's not a chance they'll follow it, formatting is for humanities majors so you'll need to catch and fix all that yourself.

That's not to say the LLMs are foolproof or magically always correct, but a lot of these style of criticisms apply just as much, if not more, to the current status quo. I don't need LLMs to be perfect, I just need them to be better than the current alternatives.

Re: LLMs corrupt your documents when you delegate

#43

Post by wtetzner »

I think the problem is that we're using LLMs to do too much of the work. We should aim to design agents that use the LLM as the thinnest possible layer to translate the natural language intent into a deterministic process, minimizing round trips to the LLM as much as possible.

Re: LLMs corrupt your documents when you delegate

#44

Post by danielvaughn »

I've spent the last few months reading a lot of AI-generated code. It's extremely difficult.

It's like how psychopaths are eerie because there's nothing behind their eyes. AI-generated code is eerie because there's nothing between the lines. Code is in some sense theory building, and when you read a humans code you can (mostly) feel their theory working in the background. LLMs have no such theory, the code is just facts strewn about. Very weird experience to try and understand it.

Re: LLMs corrupt your documents when you delegate

#45

Post by xp84 »

RevEng wrote: post #36
threethirtytwo wrote: post #28

Earlier quoted context omitted.

A human doing the same tasks as what the LLM did in the paper that the human will degrade the document further then the LLM. If the LLM is 25%, a human would degrade it probably 80% if they used the same technique as the LLM did in this paper. I'm talking about a single pass. The fact of the matter is, humans don't edit things the way it was done in the paper and neither do coding agents like claude. Think about it:…

Except that coding agents will do this at times. That's half the problem. A human will forget details and exaggerate others, but LLMs fail in spectacular ways that humans rarely would, like trying to copy a document from memory rather than one word at a time, side by side, or rewriting the whole thing just to make some simple changes. Coding agents will delete tests or return True to get them to pass - something you…

> nuke

> I don’t have to do that with any of my juniors…

For some values of “nuke,” I absolutely have had to do that with juniors in the past. Perhaps you’re referring to a single rm -r or hilarious force push or something, but undertrained and unsupervised juniors regularly introduce things like SQL injection, XSS, etc. simply because they don’t know any better yet. This isn’t saying “AI is better across the board” - I just don’t think they’re comparable, also think AI shouldn’t be used to chop the bottom 5 rungs off our career ladder. But let’s not pretend juniors can be left alone with a codebase without any worries.

Re: LLMs corrupt your documents when you delegate

#46

Post by ieieue »

timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

LLM’s are the most elaborate guessing machine man-kind has made. That’s makes it both useless and useful depending on what it is used for.

That’s it. Once you look at everything through this lense everything makes sense - especially the fact there is no underlying understanding of reasoning and creativity. I don’t care what boosters say.

Re: LLMs corrupt your documents when you delegate

#47

Post by giancarlostoro »

Twirrim wrote: post #27
timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

A coworker talks about LLMs as "bullshit" layers. Not exactly dismissing them or being derogatory about them, but emphasising that each time you feed something through an LLM, what comes out the other side may not be what you expect/want. Like that guy at the pub sharing what he'd seen online somewhere, after a few pints. Might be accurate, but carries notable risk it's not. So e.g., don't use an LLM to call an API t…

Before Claude Code my strategy in JetBrains AI was to start a new chat convo per task it yielded better output.

Re: LLMs corrupt your documents when you delegate

#48

Post by ieieue »

threethirtytwo wrote: post #26
simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically. The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have…

[flagged]

Re: LLMs corrupt your documents when you delegate

#50

Post by andrewljohnson »

LLM editing should be done to produce deterministic output.

That is, the LLM should produce a diff, and the user should accept the diff. It seems like a bad pattern to just tell the LLM to edit any long document without that sort of visibility. Same goes for prose as for code.

Post reply on HN