Live data from Hacker News

Difftastic: A diff that understands syntax

github.com

101–110 of 224 posts

Re: Difftastic: A diff that understands syntax

#101
post #85

I was interested in SemanticMerge/XMerge but when I looked they didn't have a Mac clinet and now it looks like they don't have a personal edition. I just want to buy a private license and use it locally. https://semanticmerge.com

They are requesting feedback on the pricing model for the latest revision of the technology, maybe HN could change their minds:

https://www.gmaster.io/pricing

Re: Difftastic: A diff that understands syntax

#102
post #72
post #48

Earlier quoted context omitted.

With only basic knowledge in the domain I would assume it is hard and ugly. If the file is malformed, there is almost certainly an infinite number of possible edits to make the file adhere to the grammar, hence there can not be any algorithm that just provides the one and only correct syntax tree. This in turn means that you have to come up with heuristics that identify reasonable changes which fix the file and that…

"If the file is malformed, there is almost certainly an infinite number of possible edits to make the file adhere to the grammar, hence there can not be any algorithm that just provides the one and only correct syntax tree. This in turn means that you have to come up with heuristics that identify reasonable changes which fix the file and that is probably not easy." Don't we call such heuristics "test suites"?

I don't understand that question. Given the following source file that does not parse

  var foo = bar baz
there are many ways to change it and make it parse including the following reasonable ones

  var foo = barbaz
  var foo = "bar baz"
  var foo = { bar, baz }
  var foo = bar // baz
  var foo = bar
  //var foo = bar baz
  var foo = bar * baz
  var foo = bar + baz
  var foo = bar.baz
  var foo = bar(baz)
but also unreasonable ones like

  var abc = 123
and therefore a parser that can handle malformed inputs has to make educated guesses what the input was actually supposed to look like. And don't be fooled by this simple example, imagine a long source file with deeply nested code in a language with curly braces and randomly deleting some of the braces. Now try to figure out where classes, methods, if or try statements begin and end in order to produce a [partial] syntax tree better than just giving up at the position of the first error.

Re: Difftastic: A diff that understands syntax

#103
post #53

I would love it if version control stored an AST that also includes comments and dividers (where right now we would leave an empty line) and dev machines rendered it out however they wanted. They could even change the language of keywords in addition to normal formatting.

This exact project is called JetBrains MPS.

MPS seems to be a DSL authoring tool. How would this be used to make an AST diff tool?

https://www.jetbrains.com/mps/

https://en.wikipedia.org/wiki/Abstract_syntax_tree

Re: Difftastic: A diff that understands syntax

#104
post #85

I was interested in SemanticMerge/XMerge but when I looked they didn't have a Mac clinet and now it looks like they don't have a personal edition. I just want to buy a private license and use it locally. https://semanticmerge.com

They are requesting feedback on the pricing model for the latest revision of the technology, maybe HN could change their minds: https://www.gmaster.io/pricing

OS X and Linux are "wait & see" again. That describes half of our dev team and most of the seniors.

Re: Difftastic: A diff that understands syntax

#105
post #60

Earlier quoted context omitted.

I would naively expect that this problem is easiest to solve for languages like JSON that have an unambiguous way to be pretty printed.

Indeed. One could just do `diff $(jq . $fileOne) $(jq . $fileTwo)` and you'll end up with a "nice enough" diff even if $fileOne and $fileTwo were very differently formatted.

The problem is when a file also needs to be normalized - e.g. object keys in a different order, YAML syntax expansion. It can be very useful to indicate when a JSON file is identical to another JSON file but some of the properties or array items are out of order and that requires more in-depth knowledge of the data format. Let's not mention that you could UTF-8 encode characters or write out the same character using backslash notation, numeric or boolean data that might be wrapped in a string in one file but not in another, etc. There can still be a lot of modelling and interpretation to consider when comparing data files rather than code files.

Re: Difftastic: A diff that understands syntax

#106
post #78

Earlier quoted context omitted.

Ah, well, if you're willing to accept having a frankensystem with a mix of packaged and unpackaged software, sure. ;) I used to do that, back in Slackware days. It's considered really sloppy and unmaintainable to admin a system like that. Things quickly get out of hand. That strategy _does_ work if you isolate it to a chroot or a container, but littering /usr/local with all sorts of locally compiled upstream is just…

The method I posted above doesn't write anything to /usr/local. Root isn't required. Everything is written under ~.

Whoa really?

I'm sorry, and retract my ignorant assumption! Going to try it out now.

Re: Difftastic: A diff that understands syntax

#107
post #25

Earlier quoted context omitted.

Can you give a little color on where the difficulties lie? Is it an efficiency question, or is determining "which changes" hard in the first place?

Not OP, but the docs call out some "Tricky Cases" [1]. [1] https://difftastic.wilfred.me.uk/tricky_cases.html

I’d imagine there’s some challenging judgement calls that such a tool would have to make. Like, in Go, you can reorder the members of a struct definition. In many cases this is just diff noise to reviewers. HOWEVER, it does impact the layout of the struct in memory, so it can be semantically meaningful in performance work.

Re: Difftastic: A diff that understands syntax

#108
post #5

This looks absolutely amazing. One thing I do find interesting (and a wish were different) is that only programming languages are supported, rather than data formats as well. For example, two JSON documents may be valid but formatted slightly differently, or a common task for me is comparing two YAML files. Comparing config files that have a well defined syntax and or can be abstracted into a tree (JSON, YAML, TOML,…

This isn't going to add anything to existing diff tools for JSON or YAML though. Those formats barely have any syntax highlighting or complex structures.

Re: Difftastic: A diff that understands syntax

#110
post #72

Earlier quoted context omitted.

"If the file is malformed, there is almost certainly an infinite number of possible edits to make the file adhere to the grammar, hence there can not be any algorithm that just provides the one and only correct syntax tree. This in turn means that you have to come up with heuristics that identify reasonable changes which fix the file and that is probably not easy." Don't we call such heuristics "test suites"?

I don't understand that question. Given the following source file that does not parse var foo = bar baz there are many ways to change it and make it parse including the following reasonable ones var foo = barbaz var foo = "bar baz" var foo = { bar, baz } var foo = bar // baz var foo = bar //var foo = bar baz var foo = bar * baz var foo = bar + baz var foo = bar.baz var foo = bar(baz) but also unreasonable ones like v…

My point was that test suites should give you a heuristic on what corrections are good and which are bad. A source code change that turns a test fail into a test pass should be considered an improvement.
Post reply on HN