Commits are shapshots, not diffs • Hacker News Board

Re: Commits are shapshots, not diffs

#101

Post by gnufx » Thu, Dec 17, 2020, 11:43 PM UTC

tener wrote: post #46

oblio wrote: post #9

Git is the leakiest abstraction in the history of abstractions. Diffs are a "natural" object for version control yet got doesn't actually use them and as we can see in this article, multiple git commands leak this implementation detail.

Snapshots and diffs are dual. Most version control systems operate on snapshots, but there is at least one that doesn't: Darcs. Written in Haskell, it bases it's operation on an algebra of patches. Unfortunately, while usable (and neat!) for smaller projects, Darcs has serious performance problems (due to inherent algorithmic complexity) and gets slower with certain operations. Still, one can hope it will inspire a n…

Darcs is quite usable in my experience, for medium-sized projects, at least. Say ~50MB of source with ~10 years of history. Darcs has inspired the next version of itself (version 3), but it's not clear how that will compare with Pijul.

Re: Commits are shapshots, not diffs

#102

Post by mumblemumble » Thu, Dec 17, 2020, 11:44 PM UTC

war1025 wrote: post #79

feoren wrote: post #78

Earlier quoted context omitted.

I wonder if this is why Mercurial is so much simpler and easier to understand than Git -- precisely because Git was forced to adopt a more "out there" approach. Personally I'm sad Git won out over Mercurial -- the latter is a much better technology from my experience.

> Mercurial is so much simpler and easier to understand than Git I think it must just be a mindset thing. The way git works makes perfect sense to me and I don't know why you would do it any other way. But git was already gaining significant traction when I started using version control. I can see how having built your mental models around previous version control systems and then trying to superimpose that on git wo…

It's, of course, impossible to do a practical experiment where someone doesn't learn, and therefore become biased toward, one of them first, and then learn the other after.

For my part, what I can say is, I used Mercurial for one year, and Git almost exclusively (I do occasionally touch Subversion) for a straight decade after that. And I still find myself having to consult Git's documentation far more often than I ever did with Hg. And, even now, I find myself having to un-pick minor screw-ups in Git more often than I did after only a couple months on Mercurial.

I'm pretty sure that the problem here is ultimately the UI and not the data model. No, I'm sure it is. Every time there's a conversation about Mercurial vs Git, and someone says, "Git's not that bad, all you have to do is learn the data model, and then learn all the different ad-hoc ways the UI is bound to it," my immediate thought is, "You see, that's exactly the point. The thing that's nice about Hg is that you don't have to take this extra learning step, because the data model and the UI model are one and the same."

Also, merge conflicts seem much more common in Git. I'm not sure if that's a UI problem or a data model problem; I could see it being either or both. In any case, I felt much more comfortable allowing branches to live for days in Hg. In Git, branches that survive past 24 hours seem to always result in either shaving or getting trampled by yaks.

Re: Commits are shapshots, not diffs

#103

Post by rualca » Thu, Dec 17, 2020, 11:46 PM UTC

yoz-y wrote: post #87

war1025 wrote: post #79

Earlier quoted context omitted.

> Mercurial is so much simpler and easier to understand than Git I think it must just be a mindset thing. The way git works makes perfect sense to me and I don't know why you would do it any other way. But git was already gaining significant traction when I started using version control. I can see how having built your mental models around previous version control systems and then trying to superimpose that on git wo…

Underlying technology aside, mercurial has (I think) quite objectively better cli interface. All flags are the same among commands, and all commands have their undo counterparts.

> Underlying technology aside, mercurial has (I think) quite objectively better cli interface.

I disagree. The fact that mercurial relied extensively on extensions to implent basic features, thus exposing a non-standard interface to the world, made it's mental load significantly higher than simply using a standardized (albeit debatable) interface to do standard things.

Case in point: requiring installing extensions to stash local changes.

Re: Commits are shapshots, not diffs

#104

Post by gnufx » Thu, Dec 17, 2020, 11:48 PM UTC

All this does seem to help make the argument for simple patch-based systems, i.e. Darcs and Pijul, in common with much discussion I see about git. (Their patches aren't just deltas, of course.)

Re: Commits are shapshots, not diffs

#105

Post by rualca » Thu, Dec 17, 2020, 11:48 PM UTC

dilap wrote: post #20

This is why I'm really excited about the potential of Pijul. In pijul, commits are diffs, and you avoid all of the rebase/cherry-picking craziness of Git. Still alpha software, so tons of roughs edges, but the potential is incredible. I think it'll be similar to the centralized -> distributed revolution that git ushered in.

> (...) and you avoid all of the rebase/cherry-picking craziness of Git.

What's that rebase/cherry-picking craziness?

Re: Commits are shapshots, not diffs

#106

Post by TrianguloY » Thu, Dec 17, 2020, 11:49 PM UTC

globular-toast wrote: post #96

TrianguloY wrote: post #93

When I need to explain git to someone for the first time I always start with the 'imagine you are copy/pasting your project folder, and you have multiple folders tagged with some details. Each folder is a commit'. People understand very easily how to keep copies of their work at a given time by simply coping the full folder (probably we all did this before using git). So yes, a commit is just a snapshot, a copy, it i…

Yeah. That's a perfectly valid form of version control. You could tell them to imagine renaming the directory each time with an incrementing number. That's the version number, hence version control. Merge can be explained too. Say you send the folder to your friend and that night you both make a new version. If you want to consolidate your changes you have to merge them together. I don't see why this requires commits…

In a merge without conflicts (for example you edit one file and your friend edit another) a merge could be explained as simply pasting one folder into the other, but then you need to be careful to override only one of the two files.

Merging folders is not easy if you don't know what changed, but if you do know because your friend told you 'I simply modified this file' then you know how to merge them. But this information is a diff.

In this particular example you need the new folder and the what changed information. That's what I was referring to.

Rebasing...is far more advanced to explain, yes.

Edit: or you can paste your folder into theirs, but still you need to know what you changed, your diff, so it's the same.

Re: Commits are shapshots, not diffs

#107

Post by mproud » Thu, Dec 17, 2020, 11:49 PM UTC

shapshots?

Re: Commits are shapshots, not diffs

#108

Post by kelnos » Thu, Dec 17, 2020, 11:57 PM UTC

war1025 wrote: post #79

feoren wrote: post #78

Earlier quoted context omitted.

I wonder if this is why Mercurial is so much simpler and easier to understand than Git -- precisely because Git was forced to adopt a more "out there" approach. Personally I'm sad Git won out over Mercurial -- the latter is a much better technology from my experience.

> Mercurial is so much simpler and easier to understand than Git I think it must just be a mindset thing. The way git works makes perfect sense to me and I don't know why you would do it any other way. But git was already gaining significant traction when I started using version control. I can see how having built your mental models around previous version control systems and then trying to superimpose that on git wo…

As another data point, I used CVS and later SVN for years before I used git, and I find git to be incredibly easy to use and understand. But I've been using it for over a decade, and spent time in the early days to understand how it works under the hood.

This isn't to say git has a good user interface. I don't think it does. I'm comfortable with it because I've spent many years using it, and because I expended extra effort (effort I don't think an average user should have to expend) to learn it well, and to be unafraid of making mistakes while poking around.

Re: Commits are shapshots, not diffs

#109

Post by kelnos » Fri, Dec 18, 2020, 12:10 AM UTC

DoofusOfDeath wrote: post #83

espadrine wrote: post #70

A fascinating bit of history is that the reason for this data structure was to explicitly distance git from BitKeeper. BitKeeper was used by many kernel developers, until one of them (the co-inventor of rsync, unsurprisingly) reverse-engineered its protocol, leading its proprietary owner to end its offer of a free license to Linus Torvalds and other developers. While searching for a replacement, the Linux community s…

I forget - was there another side to the story that made McVoy's actions seem more reasonable? If all I had to go on was Bryan O'Sullivan's email, I'd be tempted to draw some unflattering conclusions about McVoy's conduct.

I read a lot of the mailing list threads back when all this was going on, and I don't recall feeling any particular sympathy toward McVoy at all.

The fundamental issue I saw was that McVoy decided to foist what I consider an unconscionable license agreement upon his unpaid open source users: users were prohibited from working on another version control system without having their license to use the read-only BitKeeper client revoked. (And reverse engineering BK's protocol was also forbidden.) From the starting line I already thought he was slimy for that.

This is where Torvalds should have gotten his big honking "I told you so", but I expect he took the wrong lesson from it since he was able to whip up an acceptable alternative in git in a fairly short amount of time[0].

Anyway, McVoy was usually pretty civil in email threads about the whole thing (likely with a few angry words here and there), but I don't think that really mattered to me: IMO he was asking something unethical of his users from the start, and so I think ethically Tridge was completely in the right to reverse engineer BK's protocol and write an unencumbered client. McVoy's response to take his ball and go home was of course entirely within his rights, but was oh so childish. And, surprise, BitKeeper has since failed as a product; I believe it's now open source, and has a tiny fraction of git's user base.

[0] I think it's easy for some to take the -- IMO mistaken -- view that "being allowed to" (eye roll) use BitKeeper was a "generous" gift, because it pushed someone to build a viable open-source alternative when their back was against a wall. My view is that it was just lucky for the open source community that McVoy went up against one of the few people with the talent and motivation to out-build BitKeeper. This is one of those times RMS's preaching against using closed-source software is so spot-on, publicly, painfully correct.

Re: Commits are shapshots, not diffs

#110

Post by thaumasiotes » Fri, Dec 18, 2020, 12:11 AM UTC

fanf2 wrote: post #97

thaumasiotes wrote: post #51

Earlier quoted context omitted.

> Can you not calculate one from the other? Yes, you can. > Snapshots and diffs are just a storage implementation detail, no? If this is what you think, the headline is truly awful; git commits are stored as diffs where possible.

It’s more complicated than that! Packfiles use delta compression to store repositories more compactly, but the deltas are not at all related to commits. The objects (blobs, trees, etc.) are sorted for similarity, completely ignoring the graph of commit history and completely ignoring filenames or directory structures , then delta compression is applied. This allows git’s storage to be smaller than version control sys…

> Packfiles use delta compression to store repositories more compactly, but the deltas are not at all related to commits. The objects (blobs, trees, etc.) are sorted for similarity, completely ignoring the graph of commit history

While I understand this point, I would make two further ones:

- The storage is still diffs, and definitely not snapshots. It's just that the diffs of which a given commit consists do not all refer to the same "base". Contrasting git with a system in which commits are stored as one large diff (whereas git stores them as many small diffs) is something you can do, but it's not the same thing as contrasting git with a system in which commits are stored as snapshots. (Whereas git stores them as diffs.)

- "Sorting for similarity" isn't a thing you can do; similarity does not induce a consistent ordering.

> This allows git’s storage to be smaller than version control systems that use a storage structure that preserves history

This is worded unfortunately; git wouldn't be a version control system at all if its storage structure didn't preserve history. It does, but it doesn't mix that concern with its concern for file contents.