Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

221–230 of 235 posts

Re: LLMs corrupt your documents when you delegate

#221

Post by Kim_Bruning »

oblio wrote: post #219
Kim_Bruning wrote: post #218

Earlier quoted context omitted.

No wait, you think I'm being silly so that's why you're being a bit sarcastic back. But seriously, you can put a shebang on an english text file now (if you're sufficiently brave), or feed it through something that spits out code on the other end (so you can proof read the consequences before executing them). It's crazy, but this is 2026, and that actually ... just works. You can even do it locally, if you don't mind…

> why would you want to constrain yourself to a subset? That's not very practical. Why not use all of it? For the same reason math, physics, chemistry, etc figured out a long time ago that Koine Greek, Latin, French, German, English, etc aren't the best languages for science. Constraint gives focus, precision. If you code novels, knock yourself out.

Let's actually look at this as if I'm serious for a second? Tell me this framing really can't work.

None-exhaustively:

Python has if, for, while, def, class and first class lists, dicts , functions ;

Forth has this stack machine concept, RPN, compilation-in-the-REPL when defining new functions.

Lisp has this code is data is code concept, and CAR, CDR, first class lists (obviously ), first class functions (in some of them) ... etc.

Machine code can (theoretically) be directly expressed in logic gates.

How about a quick look at what English supports:

Conditionals, iteration, abstraction, composition, delegation, exception handling, scope, naming, modularity; intent, priority, graduated precision, analogy, context-dependence; And.. the concept of semantic triples is built in as a syntactic primitive (subject-predicate-object), so you can even do a bunch of GOFAI right off the bat.

It's weird thinking of english as a programming language. But it kind of works like one if you want to, and computers can process it now?

Re: LLMs corrupt your documents when you delegate

#222

Post by oblio »

Kim_Bruning wrote: post #221
oblio wrote: post #219

Earlier quoted context omitted.

> why would you want to constrain yourself to a subset? That's not very practical. Why not use all of it? For the same reason math, physics, chemistry, etc figured out a long time ago that Koine Greek, Latin, French, German, English, etc aren't the best languages for science. Constraint gives focus, precision. If you code novels, knock yourself out.

Let's actually look at this as if I'm serious for a second? Tell me this framing really can't work. None-exhaustively: Python has if, for, while, def, class and first class lists, dicts , functions ; Forth has this stack machine concept, RPN, compilation-in-the-REPL when defining new functions. Lisp has this code is data is code concept, and CAR, CDR, first class lists (obviously ), first class functions (in some of…

I'm not saying English (or any other natural language) is not usable. It is, since it's a more complex language than programming language. All natural languages are supersets of current programming languages.

I'm talking about the opposite problem: these supersets are ambiguous, contradictory, vague. At the end of the day the thing that is programmed needs to be clear, unambiguous and ideally concise, too (performance in its million incarnations).

So yeah, I guess you can fix the ambiguous aspect with verbosity. Just write more words until you define everything you need to define more directly when using a formal language.

I would be extremely shocked if programming wouldn't require knowing a very specific, albeit huge, domain jargon.

Re: LLMs corrupt your documents when you delegate

#223

Post by Kim_Bruning »

oblio wrote: post #222
Kim_Bruning wrote: post #221

Earlier quoted context omitted.

Let's actually look at this as if I'm serious for a second? Tell me this framing really can't work. None-exhaustively: Python has if, for, while, def, class and first class lists, dicts , functions ; Forth has this stack machine concept, RPN, compilation-in-the-REPL when defining new functions. Lisp has this code is data is code concept, and CAR, CDR, first class lists (obviously ), first class functions (in some of…

I'm not saying English (or any other natural language) is not usable. It is, since it's a more complex language than programming language. All natural languages are supersets of current programming languages. I'm talking about the opposite problem: these supersets are ambiguous, contradictory, vague. At the end of the day the thing that is programmed needs to be clear, unambiguous and ideally concise, too (performanc…

Question isn't per-se "is English a great language to write the next sorting algorithm in?" . Probably not. Rust is quicker , and cheaper to execute besides. But there's entire classes of problem that English might be more useful for.

English assumes the target is an agent with memory/state in a given context. Ambiguity, verbosity, noise is strongly reduced by means of modelling the other agent's state, then only transmitting the required state diff. The receiver decodes by comparing the diff against the other side's predicted state and updating. [1] This kind of protocol would obviously be NUTS to build from scratch if you went about it as an engineer I'd think. But we have the hardware and software preinstalled in humans , and now my 3090 can run an (imperfect, but viable) decoder

Is it useful? Yeah, I think it actually is. English is able to encode things that are ambiguous, contradictory, vague... and get useful results. Not always; maybe not even often. As you say, skill required, but the option is there. Formal languages just crash.

It's interesting is what I'd call it.

[1] see also: Clark & Brennan's grounding theory in linguistics; Predictive coding in neuroscience; Delta encoding in compression; and Theory of mind in cognitive science. They all dance around the same shape, so this is roughly accurate I think.

Re: LLMs corrupt your documents when you delegate

#224

Post by simonw »

fluidcruft wrote: post #195
wfurney wrote: post #185

Earlier quoted context omitted.

I’m interested in the “non-trivial” point as well, this seems to be a common refrain from the anti-LLM tech crowd, “LLMs aren’t good at doing anything non-trivial”, well is that really the case or is it just harder and one needs to put in more practice for more complicated tasks? I don’t have an example off hand, but I know that it’s easy to dismiss something an LLM does as trivial if your work is extremely marginal.…

On the one hand there's simonw's concept of a "frequent LLM user" and then there's the actual vast majority of people using ChatGPT web app or one of the various Office CoPilots.

I should have said "frequent, expert LLM user".

Re: LLMs corrupt your documents when you delegate

#225

Post by kristjansson »

nullsanity wrote: post #199
kristjansson wrote: post #117

Earlier quoted context omitted.

Only sort of related, but I would love to see a harness with ed as the primary file editing / reading tool. Half the bash Claude runs seems to be sed anyway, having some state persist in ed would seem to help. What does one do when a full editor consumes too much bandwidth^H tokens? Use ed, the standard editor!

I'm not sure you understand how those terminal programs are rendered - but the amount of control code data sent to Claude would be way, way more than using command line sed.

You may want to take a look at `man ed`, `info ed`, or [0]. ed is many things, but verbose is not one of them.

In particular, it's designed for the teletype era, when (a) the user would have a trace of all the commands they'd sent and output they'd received, since it was literally printed on paper, and (b) output was literally printed on paper, and so had a direct, non-negligible cost.

This is more or less exactly the situation LLMs find themselves in. they can attend to ~all the prior output in their context window, but there's a direct cost to adding new symbols to context.

We've got a tool for exactly that setting, so it would be fun to try it!

[0]: https://en.wikipedia.org/wiki/Ed_(text_editor)

Re: LLMs corrupt your documents when you delegate

#226

Post by yencabulator »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

> The modern coding agent harnesses put a LOT of work into the design of their tools for editing files. My favorite current example of that is the Claude edit suite [...]

Meanwhile, Claude Code extracting a part of a function, moving it to a different file, etc will corrupt your source code, just like the paper says. This is most noticeable as comments disappearing.

We need tooling with copy-paste/cut-paste style functionality, to avoid the LLM round-trip.

Re: LLMs corrupt your documents when you delegate

#227

Post by simonw »

Kim_Bruning wrote: post #218
oblio wrote: post #211

Earlier quoted context omitted.

> I figure english is the next coolest programming language for scripting and compilation. So far people have been writing fun little demos with it, but now people are starting to place real demands on it, and you're starting to see actual programs needing to be built. Unsurprisingly this requires a bit more craft. Perhaps that craft of using the exact subset of English has something to do with the correct selection…

No wait, you think I'm being silly so that's why you're being a bit sarcastic back. But seriously, you can put a shebang on an english text file now (if you're sufficiently brave), or feed it through something that spits out code on the other end (so you can proof read the consequences before executing them). It's crazy, but this is 2026, and that actually ... just works. You can even do it locally, if you don't mind…

> But seriously, you can put a shebang on an english text file now (if you're sufficiently brave)

That inspired me to figure out how to do exactly that:

https://til.simonwillison.net/llms/llm-shebang

  #!/usr/bin/env -S llm -f
  Generate an SVG of a pelican riding a bicycle
Thanks for the inspiration!

Re: LLMs corrupt your documents when you delegate

#228

Post by oblio »

Kim_Bruning wrote: post #218
oblio wrote: post #211

Earlier quoted context omitted.

> I figure english is the next coolest programming language for scripting and compilation. So far people have been writing fun little demos with it, but now people are starting to place real demands on it, and you're starting to see actual programs needing to be built. Unsurprisingly this requires a bit more craft. Perhaps that craft of using the exact subset of English has something to do with the correct selection…

No wait, you think I'm being silly so that's why you're being a bit sarcastic back. But seriously, you can put a shebang on an english text file now (if you're sufficiently brave), or feed it through something that spits out code on the other end (so you can proof read the consequences before executing them). It's crazy, but this is 2026, and that actually ... just works. You can even do it locally, if you don't mind…

> But seriously, you can put a shebang on an english text file now (if you're sufficiently brave), or feed it through something that spits out code on the other end (so you can proof read the consequences before executing them).

The funky thing is that it's not just English. I could vibe code in Romanian, it would probably be hilarious :-)) Probably not for whomever would have to take over the app, though.

Re: LLMs corrupt your documents when you delegate

#229

Post by Kim_Bruning »

simonw wrote: post #227
Kim_Bruning wrote: post #218

Earlier quoted context omitted.

No wait, you think I'm being silly so that's why you're being a bit sarcastic back. But seriously, you can put a shebang on an english text file now (if you're sufficiently brave), or feed it through something that spits out code on the other end (so you can proof read the consequences before executing them). It's crazy, but this is 2026, and that actually ... just works. You can even do it locally, if you don't mind…

> But seriously, you can put a shebang on an english text file now (if you're sufficiently brave) That inspired me to figure out how to do exactly that: https://til.simonwillison.net/llms/llm-shebang #!/usr/bin/env -S llm -f Generate an SVG of a pelican riding a bicycle Thanks for the inspiration!

Oh, that looks pretty clean!
Post reply on HN