Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

51–60 of 235 posts

Re: LLMs corrupt your documents when you delegate

#51

Post by CamperBob2 »

ieieue wrote: post #46
timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

LLM’s are the most elaborate guessing machine man-kind has made. That’s makes it both useless and useful depending on what it is used for. That’s it. Once you look at everything through this lense everything makes sense - especially the fact there is no underlying understanding of reasoning and creativity. I don’t care what boosters say.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning.

If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

Re: LLMs corrupt your documents when you delegate

#52

Post by jvuygbbkuurx »

devmor wrote: post #39
ekidd wrote: post #30

Earlier quoted context omitted.

Where this result is actually interesting and relevant is when a coding agent splits a large source file into multiple smaller files. Opus + Claude Code will try to recite long sections of source code from memory into each of the new files, instead of using some sort of copy/paste operation like a human would. Moving a file is a bit easier. LLMs may sometimes try to recite the file from memory. But if you tell them t…

If you’re using LLMs for agentic work it is absolutely essential that you have a robust set of tools for them to use and the correct instructions to prompt their use. The LLM will come up with stupid ways to do things, common sense doesn’t exist for AI.

Isn't this the whole reason they became viable in the last 6 months? The system prompt and harness is improving. It's less and less essential every day to roll your own.

Re: LLMs corrupt your documents when you delegate

#54

Post by embedding-shape »

jvuygbbkuurx wrote: post #52
devmor wrote: post #39

Earlier quoted context omitted.

If you’re using LLMs for agentic work it is absolutely essential that you have a robust set of tools for them to use and the correct instructions to prompt their use. The LLM will come up with stupid ways to do things, common sense doesn’t exist for AI.

Isn't this the whole reason they became viable in the last 6 months? The system prompt and harness is improving. It's less and less essential every day to roll your own.

I don't think there is a single reason. Models are improving, so are the harnesses, prompts and we who use them a lot also get more proficient and learn where they can be used effectively vs not, so lots of improvements all over the ecosystem, brought together.

Latest big change is probably how feasible local models are becoming, like Qwen 3.6 and Gemma 4, they're no longer easily getting stuck in loops and repetition, although on lower quantizations they still pretty much suck for agentic usage.

Re: LLMs corrupt your documents when you delegate

#55

Post by embedding-shape »

Kim_Bruning wrote: post #37
timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

There's a kid's game that illustrates this too: https://en.wikipedia.org/wiki/Telephone_game

Maybe more relatable to the typical HN reader: You know when the top boss tells the lower bosses stuff, who then tells the lower bosses something and once it reaches you as an IC it's all different and corrupted compared to what it initially was? LLMs have the same effect, unsurprisingly.

Re: LLMs corrupt your documents when you delegate

#56

Post by Terr_ »

CamperBob2 wrote: post #51
ieieue wrote: post #46

Earlier quoted context omitted.

LLM’s are the most elaborate guessing machine man-kind has made. That’s makes it both useless and useful depending on what it is used for. That’s it. Once you look at everything through this lense everything makes sense - especially the fact there is no underlying understanding of reasoning and creativity. I don’t care what boosters say.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning. If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

My dear sir, the entire universe is made of things that "do math without reasoning!"

It's the default, and if we're lucky we harness pieces of it to discern something we're interested in.

Re: LLMs corrupt your documents when you delegate

#57

Post by figarus314 »

CamperBob2 wrote: post #51
ieieue wrote: post #46

Earlier quoted context omitted.

LLM’s are the most elaborate guessing machine man-kind has made. That’s makes it both useless and useful depending on what it is used for. That’s it. Once you look at everything through this lense everything makes sense - especially the fact there is no underlying understanding of reasoning and creativity. I don’t care what boosters say.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning. If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

A model solving original math problems may look like human reasoning, but internally the model is choosing the next token based on what it has learned about probability around various patterns and structures. The model knows about correlations between problems, proof techniques and answer structures, and when it "reasons" it's selecting a high probability trajectory through that learned knowledge.

A calculator is different because it is not probabilistic; it executes a fixed procedure. One of these models, when doing math, is more like a learned probabilistic system that understands enough structure around mathematics that some of its high probability trajectories seem like genuine reasoning.

The difference is that when a human reasoner goes to solve a problem, they'll think "this kind of proof usually goes this way" - following an explicit rule enforcement. The model may produce the same output, and may even appear to approach it the same way, but the mechanism is a probabilistic pattern selection rather than explicit rule enforcement.

Re: LLMs corrupt your documents when you delegate

#58

Post by ekidd »

jvuygbbkuurx wrote: post #52
devmor wrote: post #39

Earlier quoted context omitted.

If you’re using LLMs for agentic work it is absolutely essential that you have a robust set of tools for them to use and the correct instructions to prompt their use. The LLM will come up with stupid ways to do things, common sense doesn’t exist for AI.

Isn't this the whole reason they became viable in the last 6 months? The system prompt and harness is improving. It's less and less essential every day to roll your own.

The models also have far more intelligence built in. For example, the pi.dev agent harness has a system prompt which fits on a single page, and includes only 4 or 5 tools. Running with a small coding model like Qwen3.6 27B, this setup is completely capable of agentic coding.

Re: LLMs corrupt your documents when you delegate

#59

Post by Forgeties79 »

timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

I was talking about this in a thread yesterday. It’s why I don’t like blogs that are just LLM generated. I don’t care how good you think it is, I don’t care that you consider a facsimile of you good enough. If I want a rote, boring LLM response, I will prompt it myself. I do not appreciate reading blogs and other assumed to be human-generated content and having somebody attempt to trick me into reading their prompt results like some annoying middleman.

I came to your blog to read what you had to say. Why are you writing a blog if you aren’t even going to write it?

Re: LLMs corrupt your documents when you delegate

#60

Post by ClikeX »

ekidd wrote: post #30
timacles wrote: post #14

Least shocking thing I've read about LLMs recently. They are essentially like that one JPEG meme, where each pass of saving as JPEG slightly degrades the quality until by the end its unrecognizable. Except with LLMs, the starting point is intent. Each pass of the LLMs degrades the intent, like in the case of a precise scientific paper, just a little bit of nuance, a little bit of precision is lost with a re-wording h…

Where this result is actually interesting and relevant is when a coding agent splits a large source file into multiple smaller files. Opus + Claude Code will try to recite long sections of source code from memory into each of the new files, instead of using some sort of copy/paste operation like a human would. Moving a file is a bit easier. LLMs may sometimes try to recite the file from memory. But if you tell them t…

> And for in-place edits, you can review "git diff" for surprises.

I don't let AI touch git anyway, and I always review the diff after it generated stuff. If it modifies my documentation, I always want to check if it messed with the text instead of just added formatting.

Post reply on HN