Live data from Hacker News

Difftastic: Syntax-aware structured diff tool

github.com

31–40 of 62 posts

Re: Difftastic: Syntax-aware structured diff tool

#31

That is a really difficult problem for more reasons than what fits in this comment :) In fact, I got my PhD studying this very problem ( https://victorcmiraldo.github.io/data/MiraldoPhD.pdf ). I did not find any description of how your diffing algorithm works nor how you represent a patch. I'd be really curious to know more.

I think the diffing is the “obvious” graph search algorithm between trees, where a “tree” is a list of atoms or trees (think lisp lists).

Basically to diff a tree of n top-level elements against one of m elements, construct a graph where nodes lie on an (n+1)x(m+1) grid. Each node (a,b) corresponds to having looked at a elements of the first and matched them to b elements of the second list. Add edges (a,b)->(a+1,b) for deletion; (a,b)->(a,b+1) for insertion; and (a,b)->(a+1,b+1) for an inner diff (ie basically this graph search problem again). Choose weights to apply to node and now find the shortest path from (0,0) to (n,m).

Re: Difftastic: Syntax-aware structured diff tool

#32
post #30

I wrote diffr [0] for that purpose; it serves me well, especially if your team makes code with long lines. In my opinion, a simple approach that does NOT make any parsing is more efficient (what about bugs in your parser? code with syntax errors? also, how fast would the parser be?) [0]: https://github.com/mookid/diffr

Many of your concerns could be alleviated by using Tree-Sitter. ( https://tree-sitter.github.io/tree-sitter/ )

Tree-sitter is great, but I find it could do a better job with broken code. This is particularly important when parsing things like C or C++ where the preprocessor makes it likely that unpreprocessed code can't be parsed anyway.

Re: Difftastic: Syntax-aware structured diff tool

#33
I personally am much more excited by “sliders” than the structure-aware diffs. Marking additions between [], it is the difference between e.g.

  handle_case [some new
    case over multiple lines
  handle_case] some existing case
  
  [  check_invariant();
  }
  
  function newFunc(){
    ...
  ]  check_invariant();
  }
And

  [handle_case some
    new case]
  handle_case old case

  [function newFunc(){
    ...
    check_invariant();
  }]

Re: Difftastic: Syntax-aware structured diff tool

#34
This seems like a really difficult issue to solve in the general case, but I found that solving it for the specific case I had was a tractable problem.

I have a need to diff the output of a query and compare it to the last time it ran, to do regression testing. Just diffing the resulting CSVs wasn't very useful, because I needed the ability to do things like ignore new columns, and report the exact column that had differences from the previous version.

I was able to do that by defining a primary key on which I could outer join the two tables. Missing or new rows would be the ones that didn't join, and then I could do a per-column comparison for each row that did join.

Re: Difftastic: Syntax-aware structured diff tool

#35

That is a really difficult problem for more reasons than what fits in this comment :) In fact, I got my PhD studying this very problem ( https://victorcmiraldo.github.io/data/MiraldoPhD.pdf ). I did not find any description of how your diffing algorithm works nor how you represent a patch. I'd be really curious to know more.

I think the diffing is the “obvious” graph search algorithm between trees, where a “tree” is a list of atoms or trees (think lisp lists). Basically to diff a tree of n top-level elements against one of m elements, construct a graph where nodes lie on an (n+1)x(m+1) grid. Each node (a,b) corresponds to having looked at a elements of the first and matched them to b elements of the second list. Add edges (a,b)->(a+1,b)…

From you description it seems like we're just computing the standard insert-delete tree-edit-distance. These tend to be slow.

This implies that the patch language only supports insertion, deletion and modification of nodes, which is a shame since refactorings, moves and duplications are also common operations in the source-code domain. Additionally, if the patch language only supports insertion, deletion and modification, the merging algorithm will perform poorly.

Re: Difftastic: Syntax-aware structured diff tool

#36
post #4

Been hoping for more of this for years. We stare at diffs all day yet we have to accommodate the computer by understanding that the parenthesis it claims was changed wasn’t actually changed, there was just another set of parentheses added. There’s of course limits to how much a diff tool can extract meaning from two pieces of content, but structure and perhaps even heuristics like “new function was added here, maybe…

you'll probably enjoy the patience diff: https://blog.jcoglan.com/2017/09/19/the-patience-diff-algori...

Re: Difftastic: Syntax-aware structured diff tool

#37
post #9

To ease the pain in conventional differs, we use a pre-commit hook to format the source code (prettier). This way we only see differences if something _actually_ changed.

I think code formatting should be mandatory and one of the first things you adopt in your project. Resist code style rule changes as much as possible, and if you do, apply them across the whole codebase in one go to avoid churn and noise in diffs down the line. And if you do make style changes, put them in a separate commit at the very least so the diffs are cleaner and code reviews are easier. In my project I use go…

you might want to check out gofumpt too, if you haven't already: https://github.com/mvdan/gofumpt

Re: Difftastic: Syntax-aware structured diff tool

#38
post #9

To ease the pain in conventional differs, we use a pre-commit hook to format the source code (prettier). This way we only see differences if something _actually_ changed.

I think code formatting should be mandatory and one of the first things you adopt in your project. Resist code style rule changes as much as possible, and if you do, apply them across the whole codebase in one go to avoid churn and noise in diffs down the line. And if you do make style changes, put them in a separate commit at the very least so the diffs are cleaner and code reviews are easier. In my project I use go…

> gofmt

gofmt is in a small set of formatters that disallows configuration and choice, it's also the first to my recollection. This is a feature because deciding on a coding standard is bike shedding. It's also extremely aggressive, and undoes pretty much any choice you may make in formatting your code; a feature, again.

Not all languages are this fortunate. Some have configuration (cargo fmt), others aren't aggressive enough (Roslyn).

Re: Difftastic: Syntax-aware structured diff tool

#39
Surely far from being as elaborate as the linked tool, but I use the following git command a few dozen times daily:

  git diff --word-diff=color --word-diff-regex='\w+'
I added two aliases to my .gitconfig, one for diff and one for show:

  [alias]
    word-show = show --word-diff=color --word-diff-regex='\\w+'
    word-diff = diff --word-diff=color --word-diff-regex='\\w+'
Those small things improved development and reviewing a lot for me!

If stuff moved around or got it's indentation changed I either add `--color-moved` and/or `-w` (ignore whitespace changes) flags to filter out extra noise.

Sometimes I need to use another regex though, e.g. a simple dot `.` for match all with no greedy +

Re: Difftastic: Syntax-aware structured diff tool

#40
I'd feel so much more motivation for checking out alternative diff tools if there was a better story for integrating them with the review tools in Github, GitLab, etc. I know there's nothing anyone can do about that— it's something the Git hosts themselves have to enable, or I have to see enough benefit in it to go to an dedicated review tool to make the bother of that worthwhile.

I believe Gerrit has a pluggable diff— is there anything more broadly on improving this story?

Post reply on HN