Live data from Hacker News

Commits are shapshots, not diffs

github.blog

131–140 of 154 posts

Re: Commits are shapshots, not diffs

#132
post #9

Git is the leakiest abstraction in the history of abstractions. Diffs are a "natural" object for version control yet got doesn't actually use them and as we can see in this article, multiple git commands leak this implementation detail.

I can't agree, because I don't work on diffs, I work on whatever the current state of the file hierarchy is. When I make changes to a file, it's not enough to know just the few lines before, I need to know what versions of code are operating dozens of lines away and in different files. Further, diffs only make sense if you have the full version of the files, pre-diff. Without that full file state, diffs could operate…

> I can't agree, because I don't work on diffs, I work on whatever the current state of the file hierarchy is. When I make changes to a file,…

This is somewhat contradictory. Of course, everybody works on states. But as you write yourself, you make changes to a file, you don't create a full new version from scratch every time you add a semicolon.

So, if you "make changes", you actually do "work on diffs".

> Further, diffs only make sense if you have the full version of the files, pre-diff. Without that full file state, diffs could operate in all sorts of incorrect ways.

That actually is an argument against Git: because Git doesn't model diffs, yet tries to simulate them, Git merges and rebases do all sort of incorrect things. For example, even the following simple example is not correctly handled in Git:

https://pijul.org/manual/why_pijul.html (see the figure "git merge vs pijul merge").

Re: Commits are shapshots, not diffs

#133
post #9

Git is the leakiest abstraction in the history of abstractions. Diffs are a "natural" object for version control yet got doesn't actually use them and as we can see in this article, multiple git commands leak this implementation detail.

I'm not sure I understand what you mean. Snapshots and diffs are different views of the same thing. You need both. It doesn't really matter which one is more natural or primitive in the implementation. When I git push, I upload my commits (i.e. snapshots). And when I git cherry-pick, I apply a diff. Both views are needed, I don't see why it's a leak.

> Snapshots and diffs are different views of the same thing.

Not completely: think of conflict resolutions for example. In patch-based systems such as Darcs and Pijul, you don't need a "rerere" command to handle that special case.

And I'm sure you'll agree that conflicts is the one critical situation where we need the best possible tool.

> When I git push, I upload my commits (i.e. snapshots). And when I git cherry-pick, I apply a diff

That's not how Git does it. Instead, Git runs 3-way merge, which doesn't do what you expect, see https://pijul.org/manual/why_pijul.html.

Re: Commits are shapshots, not diffs

#134
post #71

Earlier quoted context omitted.

To expand, since we agree. Diffs are "natural" objects for users of source control. Diffs are not natural objects for storing anything since that's not the job of the users, it's the job of the version control system. It can use ponies to store the "controlled versions" for all I (and probably other users) care.

I don't know about ponies, but for snapshots, I disagree: snapshots don't model diffs properly, and many operations in Git are essentially trying to simulate patches: - Merge and rebase both try to "replay" patches. But because Git doesn't work with patches, it actually only simulates that, using heuristic algorithms such as 3-way merge. This works most of the time, but not always (see https://pijul.org/manual/why_pi…

> Short-lived branches are another poor simulation of patches: if Git were really working on patches, you would be able to create a branch "after the fact", meaning that you would work (possibly on multiple features at the same time), and push different parts of your work separately, without having to worry about branches before you start.

You made me remember something I've been annoyed with version control systems. Which I've found hard to explain. But your comment makes me think about what doesn't come across.

The problem with version control systems is they are stand alone tools that don't capture and record refactoring. What you want is not a patch/diff. You want a record of the refactoring commands, and the result as separate things. And the ability to tag those.

  1 Tag with meta data (commit message etc)
  2 Command: Change this method name from this to that. 
  3 ChangeSet: List of 371 places where the change was made.
Consider if you have a couple of projects that rely on a framework/library. You could apply the commands to each project. And automagically get change sets for each.

Re: Commits are shapshots, not diffs

#135

Earlier quoted context omitted.

Yeah. That's a perfectly valid form of version control. You could tell them to imagine renaming the directory each time with an incrementing number. That's the version number, hence version control. Merge can be explained too. Say you send the folder to your friend and that night you both make a new version. If you want to consolidate your changes you have to merge them together. I don't see why this requires commits…

In a merge without conflicts (for example you edit one file and your friend edit another) a merge could be explained as simply pasting one folder into the other, but then you need to be careful to override only one of the two files. Merging folders is not easy if you don't know what changed, but if you do know because your friend told you 'I simply modified this file' then you know how to merge them. But this informa…

Well you just say you compare both of your new versions to the common previous version and work out how to merge them. Seeing the differences between two snapshots is something easily grasped in my experience. If you can see John changed file A and Jane changed file B then it's obvious how to merge them.

When I teach git I try to convince people that such a system of keeping snapshots is a useful thing to do. Then I introduce diffing which lets you to merges. Then show how git does that automatically.

Re: Commits are shapshots, not diffs

#136
post #4

I have never understood cherry-pick for this very reason. This helps, but I'm still confused. > The git cherry-pick command creates a new commit with an identical diff to whose parent is the current commit. How can two diffs ever be considered equivalent when they include a changed file that had different starting contents? Can they?

If I have a 100-line file and on 'main' it changes near the top, but in my 'topic' branch it changes near the bottom, then I can cherry-pick 'topic' onto 'main' and Git will resolve the diff correctly. The resulting diff or patch would only change in the line numbers for the context of the diff. This is of course a very simple example. You might hit a conflict in your "git cherry-pick" command which gives you an oppo…

> If I have a 100-line file and on 'main' it changes near the top, but in my 'topic' branch it changes near the bottom, then I can cherry-pick 'topic' onto 'main' and Git will resolve the diff correctly.

That is not true: sometimes Git will take the new lines from "topic" and merge them into the new lines from "main", see https://pijul.org/manual/why_pijul.html

> You might hit a conflict in your "git cherry-pick" command which gives you an opportunity to resolve the unexpected diff issue in an appropriate way, which ends up with a different diff than before.

Sometimes when you cherry-pick, you might not even hit a "true" conflict, but if you forgot to run "rerere", you might simply hit a previously solved conflict again.

Re: Commits are shapshots, not diffs

#137
post #92

A fascinating bit of history is that the reason for this data structure was to explicitly distance git from BitKeeper. BitKeeper was used by many kernel developers, until one of them (the co-inventor of rsync, unsurprisingly) reverse-engineered its protocol, leading its proprietary owner to end its offer of a free license to Linus Torvalds and other developers. While searching for a replacement, the Linux community s…

Isn't Mercurial different only in nomenclature? The manifest id stored in a changeset is a tree snapshot. Otherwise I don't think hg-git would work as transparently as it usually does. https://www.mercurial-scm.org/wiki/ChangeSet

They did add it; I think it wasn’t there in v0.1[0]. Nowadays, the revlog contains a sequence of either snapshots or deltas[1], a bit like video codecs I-frame and P-frame, except the heuristic is how much data must be read to check out the file; so most are likely deltas.

[0]: https://lore.kernel.org/lkml/42692470.9050605@tmr.com/T/

[1]: http://hgbook.red-bean.com/read/behind-the-scenes.html

Re: Commits are shapshots, not diffs

#138
post #71

Earlier quoted context omitted.

To expand, since we agree. Diffs are "natural" objects for users of source control. Diffs are not natural objects for storing anything since that's not the job of the users, it's the job of the version control system. It can use ponies to store the "controlled versions" for all I (and probably other users) care.

I don't know about ponies, but for snapshots, I disagree: snapshots don't model diffs properly, and many operations in Git are essentially trying to simulate patches: - Merge and rebase both try to "replay" patches. But because Git doesn't work with patches, it actually only simulates that, using heuristic algorithms such as 3-way merge. This works most of the time, but not always (see https://pijul.org/manual/why_pi…

So... we're agreeing? :-)

I was criticizing Git for not using diffs/patches as a regular version control user would expect.

Re: Commits are shapshots, not diffs

#139
post #46
post #9

Git is the leakiest abstraction in the history of abstractions. Diffs are a "natural" object for version control yet got doesn't actually use them and as we can see in this article, multiple git commands leak this implementation detail.

Snapshots and diffs are dual. Most version control systems operate on snapshots, but there is at least one that doesn't: Darcs. Written in Haskell, it bases it's operation on an algebra of patches. Unfortunately, while usable (and neat!) for smaller projects, Darcs has serious performance problems (due to inherent algorithmic complexity) and gets slower with certain operations. Still, one can hope it will inspire a n…

I wonder if you could do the same thing video codecs does. Diffs, with the occasional keyframe.

Re: Commits are shapshots, not diffs

#140

Earlier quoted context omitted.

I don't quite agree with the author that you need to understand that commits are implemented as snapshots, but yes, you need to understand quite a bit of how git works behind the scenes in order to use it effectively. This is because git tries to hide more than it should, which leads to confusion when things Go Wrong.

I'd also say that when I've worked through people's confusion surrounding git (and other complex systems, it is because they are attempting to model the system as simpler than it actually has to be. They're not thinking of cases the system must handle, and in the absence of that (essential) complexity, they come up with an incorrect mental model that they believe works (as they're not aware of the counter-examples to…

> They're not thinking of cases the system must handle

That's not their fault; git's CLI attempts to abstract away the details and presents a falsely simple view of the world with "checkout", "add", "commit", and "push". This works for a while, until it doesn't, and then you have to face the complexity of the graph all at once. If I was going to redesign its CLI, I'd make the graph as transparent as possible.

Post reply on HN