Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs corrupt your documents when you delegate

arxiv.org

Post reply on HN
Print view View original

81–90 of 235 posts

Re: LLMs corrupt your documents when you delegate

#81

Post by visarga »

figarus314 wrote: post #57
CamperBob2 wrote: post #51

Earlier quoted context omitted.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning. If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

A model solving original math problems may look like human reasoning, but internally the model is choosing the next token based on what it has learned about probability around various patterns and structures. The model knows about correlations between problems, proof techniques and answer structures, and when it "reasons" it's selecting a high probability trajectory through that learned knowledge. A calculator is dif…

You talk as if problem solving is a supervised (imitation) learning problem. No, it is a reinforcement learning problem, models learn by solving problems and getting rated. They generate their own training data. Optimal budget allocation is 1/3 cost pre-training, 1/3 for RL, and 1/3 on inference.

Re: LLMs corrupt your documents when you delegate

#82

Post by tommyage »

ACCount37 wrote: post #40
lelanthran wrote: post #32

Earlier quoted context omitted.

Are we comparing humans to LLMs or human written software to LLMs? The whole point of creating software to do things used to be getting things done more accurately and consistently.

No. The whole point of creating software is getting things done . "More accurately and consistently" was merely downstream from what capabilities were natural for machine logic and hard algorithms. Now, we're just spoiled for choice. We have hard algorithm software where we want to do things that benefit for accurate, consistent, highly deterministic behavior - and we have soft algorithm AI for when we want to do thi…

All of software is hard-coded algorithm.

If you differ between AI source code and engineer source code say so. "Getting things done" is a business need. Which things get translated to a deterministic language executable by a computer is code.

There are entire languages dedicated for lesser engineers/domain experts to formulate business requirements.

Anyhow; What's your point? That we received a framework for "soft algorithms" where the output does not need to be correct and deducible? What's even the point of putting it into software. Just forward your input to the reader and let him judge on its own.

Re: LLMs corrupt your documents when you delegate

#83

Post by leptons »

danielvaughn wrote: post #44

I've spent the last few months reading a lot of AI-generated code. It's extremely difficult. It's like how psychopaths are eerie because there's nothing behind their eyes. AI-generated code is eerie because there's nothing between the lines. Code is in some sense theory building, and when you read a humans code you can (mostly) feel their theory working in the background. LLMs have no such theory, the code is just fa…

My company is moving to a workflow where we only write Jira tickets, the LLM writes all the code and submits a PR. Then we are supposed to review the code the LLM wrote.

I'm looking for a new job.

Re: LLMs corrupt your documents when you delegate

#84

Post by alansaber »

simonw wrote: post #20

I'm suspicious of their results with regards to tool usage. It's unsurprising that round-tripping long content through an LLM results in corruption. Frequent LLM users already know not to do that. They claim that tool use didn't help, which surprised me... but they also said: > To test this, we implemented a basic agentic harness (Yao et al., 2022) with file reading, writing, and code execution tools (Appendix M). We…

The incomprehensible methodology due to resource constraints or straight up for simplicity's sake make these papers worthless unfortunately

Re: LLMs corrupt your documents when you delegate

#86

Post by oldsecondhand »

CamperBob2 wrote: post #51
ieieue wrote: post #46

Earlier quoted context omitted.

LLM’s are the most elaborate guessing machine man-kind has made. That’s makes it both useless and useful depending on what it is used for. That’s it. Once you look at everything through this lense everything makes sense - especially the fact there is no underlying understanding of reasoning and creativity. I don’t care what boosters say.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning. If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

> If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

Logic is just syntactic manipulation of formulas. By the early 90s logical reasoning was pretty much solved with classical AI (the last building block being constraint logic programming).

Re: LLMs corrupt your documents when you delegate

#87

Post by alansaber »

andrewljohnson wrote: post #50

LLM editing should be done to produce deterministic output. That is, the LLM should produce a diff, and the user should accept the diff. It seems like a bad pattern to just tell the LLM to edit any long document without that sort of visibility. Same goes for prose as for code.

This gets skipped because continual approvals break up user flow so we let LLMs make a few hundred line diffs then a user does a bulk review, and can just revert all/partially. It's naieve to assume user will review every LOC in every instance.

Re: LLMs corrupt your documents when you delegate

#88

Post by leptons »

pyrolistical wrote: post #9
adampunk wrote: post #8

LLMs will make mistakes on every turn. The mistakes will have little to no apparent connection to "difficulty" or what may or may not be prevalent in the training data. They will be mistakes at all levels of operation, from planning to code writing to reporting. Whether those mistakes matter and whether you catch them is mostly up to you. I have yet to find a model that does not make mistakes each turn. I suspect tha…

As a human I make typos all the time

The LLM makes typos for me all the time using AI autocomplete. It's caused a lot of frustration while coding, because it makes mistakes that I would not. When it does help, it's great, but the errors waste as much time as the LLM saves me. Even using agentic coding, AI is mostly break-even for me.

Re: LLMs corrupt your documents when you delegate

#89

Post by CamperBob2 »

oldsecondhand wrote: post #86
CamperBob2 wrote: post #51

Earlier quoted context omitted.

I don't know what a "booster" is, but if a model can solve original math problems, then it's reasoning. If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI.

> If you can come up with a way to do math without reasoning, that would be, in a sense, even more interesting than AI. Logic is just syntactic manipulation of formulas. By the early 90s logical reasoning was pretty much solved with classical AI (the last building block being constraint logic programming).

So you'll be able to show me the early-90s era program that can solve original IMO-level problems when supplied with the plaintext questions. Right?

Re: LLMs corrupt your documents when you delegate

#90

Post by CamperBob2 »

XMPPwocky wrote: post #71
figarus314 wrote: post #57

Earlier quoted context omitted.

A model solving original math problems may look like human reasoning, but internally the model is choosing the next token based on what it has learned about probability around various patterns and structures. The model knows about correlations between problems, proof techniques and answer structures, and when it "reasons" it's selecting a high probability trajectory through that learned knowledge. A calculator is dif…

> The difference is that when a human reasoner goes to solve a problem, they'll think "this kind of proof usually goes this way" - following an explicit rule enforcement. How is this different from "probabilistic pattern selection"?

Because... it's just different, that's all! OK?
Post reply on HN