Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

141–150 of 235 posts

Re: LLMs corrupt your documents when you delegate

#141

Post by leptons »

8note wrote: post #108
leptons wrote: post #83

Earlier quoted context omitted.

My company is moving to a workflow where we only write Jira tickets, the LLM writes all the code and submits a PR. Then we are supposed to review the code the LLM wrote. I'm looking for a new job.

that doesnt seem particularly horrible, as long as you as the engineer can still go change things in the code package and surrounding infrastructure to improve the output, and make sure that the agent is actually making the right stuff the first time you see the outputs eg. setting up better feedback loops, improving CI/CD, breaking changes up at the right scale, etc. you i assume also can then put in more work up fr…

The change is turning me from someone who writes pretty good reliable code, to someone who has to read and review pretty bad code. If you think this is an improvement, you're nuts.

It is inserting a pretty unreliable middle-man know for errors and hallucinations, that often just goes down and stops working for reasons we can't control into a workflow that has worked well for a decade, and we're paying extra to really break-even on the time spent creating new code.

Just because "everyone else is doing it". Not because it's proving to be a boon in productivity.

Re: LLMs corrupt your documents when you delegate

#145

Post by ActionHank »

threethirtytwo wrote: post #26
simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically. The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have…

> a human will DO worse then a 25% degradation.

* than

Re: LLMs corrupt your documents when you delegate

#146

Post by ActionHank »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

It could also be that much like most large orgs now you've made LLMs your entire personality, so you don't see the inherent bias.

Most LLM users who are not touching code are certainly not going to be using a harness. They're going to take all the documents, slam all those tokens into the context window, see they have only used 500k out of their 1M tokens and say "summarize".

Re: LLMs corrupt your documents when you delegate

#147

Post by genxy »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

Yeah, this is a bit of a strawman of an LLM task.

On editing tasks, one should only allow programmatic editing commands, the text shouldn't flow through the LLM at all. The LLM should analyze the text and emit commands to achieve a feedback directed goal.

Re: LLMs corrupt your documents when you delegate

#149

Post by threethirtytwo »

jrflowers wrote: post #130
threethirtytwo wrote: post #126

Earlier quoted context omitted.

That's the point bro. I am saying this Experiment makes no sense. Humans don't do that. And Claude doesn't edit documents like that. Because it makes no sense. The point is saying that the Experiment itself is not helpful here.

It is, in fact, pretty common for people to throw a document at a language model along with a “make it more gooder” prompt.

That was true maybe 7 months ago. This is no longer the case. Harnesses use all kinds of tooling to edit things now.

Re: LLMs corrupt your documents when you delegate

#150

Post by threethirtytwo »

ActionHank wrote: post #145
threethirtytwo wrote: post #26

Earlier quoted context omitted.

People love to interpret the results in the most negative way possible because it's a threat to their occupation and identity. I refer to HN specifically. The fact of the matter is, if you want to edit a document by reading the document and then regurgitating the entire document with said edits... a human will DO worse then a 25% degradation. It's possible for a human to achieve 0% degradation but the human will have…

> a human will DO worse then a 25% degradation. * than

See that’s an example of degradation by a human. Not even an LLM wil make that kinda mistake.
Post reply on HN