Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are diffs still useful for AI-assisted code changes?

news.ycombinator.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#11

Post by veunes »

I totally get the fear regarding probabilistic changes being reviewed by probabilistic tools. It's a trap. If we trust AI to write the code and then another AI to review it, we end up with perfectly functioning software that does precisely the wrong thing.

Diffs are still necessary, but they should act as a filter. If a diff is too complex for a human to parse in 5 minutes, it’s bad code, even if it runs. We need to force AI to write "atomically" and clearly; otherwise we're building legacy code that's unmaintainable without that same AI

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#12

Post by veunes »

nuky wrote: post #7
DiabloD3 wrote: post #3

You know there are other kinds of diffs, right? Its common to change git's diff to things like difftastic, so formatting slop doesn't trigger false diff lines. You're probably better off, FWIW, just avoiding LLMs. LLMs cannot produce working code, and they're the wrong tool for this. They're just predicting tokens around other tokens, they do not ascribe meaning to them, just statistical likelihood. LLM weights thems…

Yeah difftastic and similar tools help a lot with formatting noise really. My question is slightly orthogonal though: even with a cleaner diff, I still find it hard to quickly tell whether public API or behavior changed, or whether logic just moved around. Not really about LLMs as reviewers — more about whether there are useful deterministic signals above line-level diff.

The tools exist, they're just rarely used in web dev. Look into ApiDiff or tools using Tree-sitter to compare function signatures. In the Rust/Go ecosystem, there are tools that scream in CI if the public contract changes. We need to bring that rigor into everyday AI-assisted dev. A diff should say "Function X now accepts null", not "line 42 changed"

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#13

Post by veunes »

uhfraid wrote: post #4

> How do you review large AI-assisted refactors today? just like any other patch, by reading it

Reading works when you generate 50 lines a day. When AI generates 5,000 lines of refactoring in 30 seconds, linear reading becomes a bottleneck. Human attention doesn't scale like GPUs. Trying to "just read" machine-generated code is a sure path to burnout and missed vulnerabilities. We need change summarization tools, not just syntax highlighting

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#14

Post by nuky »

veunes wrote: post #13
uhfraid wrote: post #4

> How do you review large AI-assisted refactors today? just like any other patch, by reading it

Reading works when you generate 50 lines a day. When AI generates 5,000 lines of refactoring in 30 seconds, linear reading becomes a bottleneck. Human attention doesn't scale like GPUs. Trying to "just read" machine-generated code is a sure path to burnout and missed vulnerabilities. We need change summarization tools, not just syntax highlighting

This is exactly the gap I'm worried about. human review still matters, but linear reading breaks down once the diff is mostly machine-generated noise. Summarizing what actually changed before reading feels like the only way to keep reviews sustainable.

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#15

Post by nuky »

veunes wrote: post #11

I totally get the fear regarding probabilistic changes being reviewed by probabilistic tools. It's a trap. If we trust AI to write the code and then another AI to review it, we end up with perfectly functioning software that does precisely the wrong thing. Diffs are still necessary, but they should act as a filter. If a diff is too complex for a human to parse in 5 minutes, it’s bad code, even if it runs. We need to…

Agreed - that trap is very real. The open question for me is what we do when atomic, 5min readable diffs are the right goal but not realistically achievable always. My gut says we need better deterministic signals to reduce noise before human review. Not to replace it.

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#16

Post by nuky »

csomar wrote: post #10

I'm working on a similar tool ( https://codeinput.com/products/merge-conflicts/online-diff ), specifically focusing on how to use the diff results. For semantic parsing, I think the best option available right now is Tree-sitter ( https://tree-sitter.github.io/tree-sitter ), which has decent WASM support. If this interests you, feel free to shoot me an email. I'm always looking to connect with other devs who want to…

Oh yeah tree-sitter it's a great foundation for semantic structure.

What I'm exploring is more about what we do with that structure once someone/smth starts generating thousands of changed lines: how to compress change into signals we can actually reason about.

Thank you for sharing. I'm actually trying your tool right now - it looks really interesting. Happy to exchange thoughts.

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#17

Post by csomar »

nuky wrote: post #16
csomar wrote: post #10

I'm working on a similar tool ( https://codeinput.com/products/merge-conflicts/online-diff ), specifically focusing on how to use the diff results. For semantic parsing, I think the best option available right now is Tree-sitter ( https://tree-sitter.github.io/tree-sitter ), which has decent WASM support. If this interests you, feel free to shoot me an email. I'm always looking to connect with other devs who want to…

Oh yeah tree-sitter it's a great foundation for semantic structure. What I'm exploring is more about what we do with that structure once someone/smth starts generating thousands of changed lines: how to compress change into signals we can actually reason about. Thank you for sharing. I'm actually trying your tool right now - it looks really interesting. Happy to exchange thoughts.

Feel free to shoot me an email (your email is not visible on your profile).

Re: Ask HN: Are diffs still useful for AI-assisted code changes?

#18

Post by uhfraid »

veunes wrote: post #13
uhfraid wrote: post #4

> How do you review large AI-assisted refactors today? just like any other patch, by reading it

Reading works when you generate 50 lines a day. When AI generates 5,000 lines of refactoring in 30 seconds, linear reading becomes a bottleneck. Human attention doesn't scale like GPUs. Trying to "just read" machine-generated code is a sure path to burnout and missed vulnerabilities. We need change summarization tools, not just syntax highlighting

Whether you or someone/something else wrote it is irrelevant

You’re expected to have self-reviewed and understand the changes made before requesting review. You must to be able to answer questions reviewers have about it. Someone must read the code. If not, why require a human review at all?

Not meeting this expectation = user ban in both kernel and chromium

Post reply on HN