Live data from Hacker News

Supercharging the Git Commit Graph

blogs.msdn.microsoft.com

71–80 of 87 posts

Re: Supercharging the Git Commit Graph

#71

Earlier quoted context omitted.

Rebasing is fundamental to a proficient local workflow. Interactive rebasing with git rebase -p lets you craft and recraft your commits easily until they are readable and tell a story. The sooner a person can rebase, the sooner they can use Git. It’s not really possibly to proficiently or pleasantly use Git without rebasing.

I very strongly disagree with that. I'd rather someone be proficient with the staging index, and maybe even git stash, than rebasing. Maybe, if they are feeling fancy, `git add --interactive`, `git add --patch`, and/or `git commit --amend`. Rebasing is a fascinating footgun. I'd rather a messy story that includes details of how someone screwed up, then fixed their mistakes, than an entire branch I need to cherry pick…

> I'd rather a messy story that includes details of how someone screwed up

Strongly disagree. I am mostly interested in reviewing your best work, and ensuring that it is correct, as much as I enjoy hearing the story about how you got there. When we spend a lot of time on specs and ensuring they are correct, it's mostly important that I can review your change and compare it to the spec to ensure correctness. A commit that was wrong, but was corrected before it ever went to prod, doesn't really help me at all during code reviews.

I agree that 'git stash' and 'git commit --amend' are also important skills to have, but I prefer to make small, incremental commits that are obviously correct changes in isolation, even if they are not very interesting, so long as they are moving something toward the goal. Then I'll condense them into commits that look something like "the whole change I intended to make" – but that's maybe once it's fully formed and the tests are actually passing.

I don't like to keep a lot of small commits around, again selfishly and simply because they can make rebasing harder. It is telling how many things I do in service of keeping it easy to rebase a changeset. It's not an unobtrusive feature and there's definitely merit to the argument that it's a nice gun to shoot yourself in the foot with.

There is a happy medium between squashing all commits within a feature branch, and keeping every "boring, but obviously correct" incremental change intact forever. A change is only interesting in isolation if it might be reverted, or if I read it in isolation and for whatever reason strongly believe that I might ever want to read it in isolation again.

Re: Supercharging the Git Commit Graph

#72
post #49

Earlier quoted context omitted.

What I'm saying is, the iteration step from "existing dataset" (what we have today) to "faster data traversal" (what the article proposes) is a custom file with a custom format on one side, and the appropriate query/index on the other side; one is definitely more understandable, portable and maintainable than the other.

Except that SQL has never had great DAG data structures, queries, nor indexes. You can model a DAG in a relational database, and you can non-standard SQL extensions to get some decent but not great recursive queries to do some okay semi-poorly indexed graph work, but having maintained databases like that at various times that all gets to be just as much a "custom file with a custom format" as dependent on database ve…

Fossil itself is stored entirely inside a SQLite db and only uses it to do everything it needs; if Fossil can do it, any VCS can do it. In fact, there is a whole section on that point in the official SQLite page (https://www.sqlite.org/lang_with.html#rcex2).

I'm not saying SQL is the best way to store and query DAGs; any graph database would be better. All I'm saying is that SQL is probably better at designing and maintaining a solution than what git does with its custom file format and custom code.

I'm only comparing what the pile-of-files that git currently is and a full-fledged SQL database. None is perfect, but one feels overall easier than the other.

Re: Supercharging the Git Commit Graph

#73
post #44
post #41

Earlier quoted context omitted.

This is about having a single, unified interface for all operations. This is all explained in great details by SQLite itself at https://sqlite.org/appfileformat.html

As usual with the SQLite / Fossil developer argumentation, it just seems very biased and far-fetched. Just one example: > Pile-of-Files Formats. Sometimes the application state is stored as a hierarchy of files. Git is a prime example of this, though the phenomenon occurs frequently in one-off and bespoke applications. A pile-of-files format essentially uses the filesystem as a key/value database, storing small chunk…

Your points are valid, especially considering that this page is explaining the benefits (for the author) of using SQLite as a generic application file format; however we're only talking about git here, and my usage of git is limited to "git some-command", sometimes "git some-command | grep foobar", and most of the time I'm in a GUI anyway. I'm not grepping the git objects directly, so whether I use a git subcommand or a sql subcommand won't make any difference to me. The real advantages of using sql subcommands for me are:

- I could probably plug that into something else with more ease than something that is git-specific - I have more flexibility for querying out of the box, without learning the specifics of each subcommand. The full SQL language is there at my disposal for outputting exactly what I need

Re: Supercharging the Git Commit Graph

#74
post #70

Earlier quoted context omitted.

> other branches should clearly flow their merges only one way I tend to disagree on this as well. The best person to merge a conflict is the developer creating the conflict in the first place, as soon or nearly as soon as they create the conflict, because they are most likely to know why the conflict exists in the first place. "Merge early, merge often." Delaying "reverse" merges until the last possible second means…

> you also aren't recording a history of the merge conflicts you saw in the rebase This is a good point. Mistakes are made during merge conflict resolution. But if you are tracking your upstream when you develop a long-lived feature branch, and rebasing when there are changes to the base, and actually comparing your rebased feature branches to the version that you had before you force push over the old remote version…

I still think you are better off preserving every merge point-in-time when they were made than ever rebasing feature branches. Because yes, every rebase is an opportunity for mistakes to go unnoticed, and there's no "rebase log" to try to unwind a mistake weeks or months later.

Also, every rebased branch is its own likely source of merge conflicts for people working on "differently rebased" versions of the same code.

> The most valuable skill we are learning from Git is how to avoid merge conflicts altogether, and it tends to be more a human problem than a technological issue. (You don't avoid merge conflicts with some special commit strategy, you do it by ensuring that two people are not actively changing the same part of the codebase unless it's absolutely necessary.)

Definitely. Communication is a key. It's also why I think "merge early, merge often" works better, because it forces communication as early and often as possible ("I'm seeing a merge conflict with this work you are doing, can you explain it to me?"). Keep branches as short-lived as possible, and try to avoid "separate but equal" work where you can't comingle features and have to intentionally fence your branches from integrating with each other. Merge feature branches between each other, even, to keep communication up. Find tools like feature flags that allow you to "ship" to master "unfinished" work faster to save yourself from trying to integrate long running branches after the fact. (If you are going to let "marketing" choose which features "ship" in a version, it is much nicer to do so by flipping flags in a dashboard somewhere, maybe even one Marketing can use themselves, than to try to furiously merge long-running feature branches at release time.)

Re: Supercharging the Git Commit Graph

#75
post #70

Earlier quoted context omitted.

> you also aren't recording a history of the merge conflicts you saw in the rebase This is a good point. Mistakes are made during merge conflict resolution. But if you are tracking your upstream when you develop a long-lived feature branch, and rebasing when there are changes to the base, and actually comparing your rebased feature branches to the version that you had before you force push over the old remote version…

I still think you are better off preserving every merge point-in-time when they were made than ever rebasing feature branches. Because yes, every rebase is an opportunity for mistakes to go unnoticed, and there's no "rebase log" to try to unwind a mistake weeks or months later. Also, every rebased branch is its own likely source of merge conflicts for people working on "differently rebased" versions of the same code.…

+1 for feature flags. We have struggled to manage long-lived changes (our current iteration has been going on for nearly a year.)

The amount of uhh "pucker" I'll feel when we release next month, and certain pieces of code are touching prod for the first time, is much higher than I'd like.

The only reason I can sleep at night while spending a whole year with only hotfixes and minor feature addons for the last release going to prod, is because we've spent much too much time at all layers of the testing pyramid, and every time I've made a change that should probably break some tests, it actually breaks a few more tests than I expected (so I know that our coverage is pretty good.) Every time we write a brand new feature, it absolutely gets a matching Cucumber feature test in plain english that we run before and after every merge. And that's not even to mention unit testing.

So we can be reasonably confident that everything we've ever promised, is still true. Our test suite takes over an hour to run at this point, which is way longer than it should. But it helps us catch the bugs, and well before they start to have "piled up."

I'd feel marginally better if we knew what code doesn't run in prod because we could turn it on and off, and see those feature tests failing (or simply exclude them, since they would also be tagged with the feature label.)

It's interesting that some people feel feature flags are complicated enough to make "Feature flags as-a-Service" businesses a thing, like LaunchDarkly. I looked at what they are offering and thought it would serve us well, but we haven't actually started using feature flags for anything.

Re: Supercharging the Git Commit Graph

#76
post #71

Earlier quoted context omitted.

I very strongly disagree with that. I'd rather someone be proficient with the staging index, and maybe even git stash, than rebasing. Maybe, if they are feeling fancy, `git add --interactive`, `git add --patch`, and/or `git commit --amend`. Rebasing is a fascinating footgun. I'd rather a messy story that includes details of how someone screwed up, then fixed their mistakes, than an entire branch I need to cherry pick…

> I'd rather a messy story that includes details of how someone screwed up Strongly disagree. I am mostly interested in reviewing your best work, and ensuring that it is correct, as much as I enjoy hearing the story about how you got there. When we spend a lot of time on specs and ensuring they are correct, it's mostly important that I can review your change and compare it to the spec to ensure correctness. A commit…

A point I've tried to make is that we don't need a "happy medium", we can keep the mess that the sausage was made from and still get the sausage final product.

We have the power of a DAG to both record and hide all the mess. I can take my messiest repository and with a couple DAG traversal options still get that clean "story" back out of it. On the flipside with rebase you are absolutely erasing history and there is no way to dig deeper into any information lost or discarded in that rebase. I can give you what you want to see from my repositories, but you can't give me what I sometimes want to see from yours.

Merge commits are a great place to tell the story of "the whole change I intended to make". A good PR system even encourages and automates exactly that. (It also makes it easier to review the branch as a whole, or incrementally, or what have you in between to suit your needs/interests.)

I don't outlaw rebasing locally. There are absolutely cases in which a local rebase is necessary, and if a developer feels comfortable with rebasing I'm not going to stop them from doing it in a branch they control.

But I also don't encourage rebasing. I'd rather see how the sausage was made, ugly as it was, as it does tell a story even if you may not find it an "interesting" story, and sometimes in code archaeology or debugging nightmares you do need to dive into tiny incremental trivia.

Re: Supercharging the Git Commit Graph

#77
post #71

Earlier quoted context omitted.

> I'd rather a messy story that includes details of how someone screwed up Strongly disagree. I am mostly interested in reviewing your best work, and ensuring that it is correct, as much as I enjoy hearing the story about how you got there. When we spend a lot of time on specs and ensuring they are correct, it's mostly important that I can review your change and compare it to the spec to ensure correctness. A commit…

A point I've tried to make is that we don't need a "happy medium", we can keep the mess that the sausage was made from and still get the sausage final product. We have the power of a DAG to both record and hide all the mess. I can take my messiest repository and with a couple DAG traversal options still get that clean "story" back out of it. On the flipside with rebase you are absolutely erasing history and there is…

Your argument has merit, but I think we can agree to disagree. I'm okay with erasing some history sometimes.

The main point I'm trying to make is that developers should not have to feel compelled by some unspoken pressure to make fewer commits, knowing that once they are made, some day, someone will be stuck reading or replaying them. Commit history is malleable, and unpublished commits are mostly free to be thought of as exactly like soup; free to stir at any time, and ladle into portions as needed.

Sometimes I read a commit in a chain of commits that I made and, though it was interesting enough to commit for some reason, I can immediately tell that the moment I've just replayed will never be a point in time that I'll ever want to replay again. I can rebase in that moment and potentially save repeating some wasted time for "future-me."

Sometimes I even know at the time of making the commit that it will not be interesting ever again, but that at that moment, I'm less likely to make an error if I just `git add -p` the "done part," commit that, and keep the WIP unstaged. Maybe the change is so boring that I'll actually write "squash" or "fixup" as the commit message, and then I'll be sure to follow my own advice some time soon, when it's not so interruptive to my flow, and I will squash that commit before it gets published or merged.

I love descriptive commit messages, but I also love totally obvious, eminently readable code. And I feel like sometimes the message I need to get out to developer members of my team is that just one "readable" or patently obvious block of code is easily worth ten descriptive commit messages.

There also may be times when "nobody is reading your commit history" is a little demoralizing but also exactly what a developer needs to hear. "We don't need a novel about each commit, write just what you need to and spend that time you saved making the code itself actually better."

Re: Supercharging the Git Commit Graph

#78
post #72

Earlier quoted context omitted.

Except that SQL has never had great DAG data structures, queries, nor indexes. You can model a DAG in a relational database, and you can non-standard SQL extensions to get some decent but not great recursive queries to do some okay semi-poorly indexed graph work, but having maintained databases like that at various times that all gets to be just as much a "custom file with a custom format" as dependent on database ve…

Fossil itself is stored entirely inside a SQLite db and only uses it to do everything it needs; if Fossil can do it, any VCS can do it. In fact, there is a whole section on that point in the official SQLite page ( https://www.sqlite.org/lang_with.html#rcex2 ). I'm not saying SQL is the best way to store and query DAGs; any graph database would be better. All I'm saying is that SQL is probably better at designing and…

But you are also almost intentionally confusing the SQL standard here in your comment with the SQLite implementation (a de facto standard, of a sort, but not a recognized standard by any body of peers to my knowledge) with SQLite's particular binary format (which does change between versions even). That is a custom file format with custom code. Certainly it is very portable custom code, as SQLite is open source and ported to a large number of systems, but just because it is related to the SQL standards doesn't gift it the benefit of being an SQL standard in and of itself.

The SQL standards define a query language, not a storage format. There are SQL databases that themselves optimize their internal storage structures into "piles of files". In fact, most have at one point or another. SQLite is an intentional outlier here; it's part of why SQLite exists.

There's nothing stopping anyone from building an SQL query engine that executes over a git database, for what that is worth. Because you can't execute SQL queries against it today doesn't really say anything at all about whether or not git's database storage format is insufficient or not.

All of that is also before you even start to get into the weeds about standards compliance in the SQL query language itself and how very little is truly compliant between database engines, as they all have slightly different dialects due to historic oddities. Or the weeds that there's never been a good interchange format between SQL database storage formats other than overly verbose DDL and INSERT statement dumps. That again are sometimes subject to compatibility failures if trying to migrate between database engines, due to dialectal differences. Including what should be incredibly fundamental things like making sure that foreign key relationships import and index correctly, without data loss or data security issues, because even some of that is dialectal and varies between engines (drop keys, ignore keys, read keys, make sure everything is atomically transacted to the strongest transaction level available in that particular engine, etc).

Git's current pile of files may not be better than "a full-fledged SQL database", that's a long and difficult academic study to undertake, but a "a full-fledged SQL database" isn't necessarily the best solution just because it has a mostly standard query language, either.

Re: Supercharging the Git Commit Graph

#79
post #64
post #22

Earlier quoted context omitted.

> git sort of sucks for monorepos. Microsoft works around several of the issues there by using GVFS. Also, at Microsoft scale, everything "sort of sucks", there's just no silver bullets. You take one of the least bad options and put all the effort you can towards making it work as well as you can.

No. Everything only "sort of sucks" at "Microsoft scale" if you're willing to blame it on "scale." At "Microsoft scale," you have the resources to purpose build anything you need from scratch for any scale you're working at... therefore if anything "sort of sucks" it's because: a) It's not worth the money/resources. (The "let it suck" approach) or b) Nobody cares (The "acceptance that we suck, at 'scale'" approach)

... and I'm going to belabor this, because I think it's important.

There's a fair bit of excitement around the new, exciting, open source friendly Microsoft with built-in Linux kernel emulation, an embrace of git and a huge release of open source tools...

Honestly though, I'm not buying it, the issue I've had with Microsoft over the years doesn't just stem from the shitty software or aggressive business practices... it's the deep rooted culture of mediocrity it promotes.

People who belong to that church believe it's ok to build shitty software because doing it right is too hard. Why root-cause an issue when you can just script reboots? It's too hard anyway, we're at "Microsoft scale."

The rise of the internet and the companies that grew around it showed us that if you have a culture of giving a shit, you can build really complex things "at scale" that aren't complete shit.

I worry that the new Microsoft will be a different kind of trojan horse for the OSS world. It won't be "embrace, extend, extinguish" it will be more like a social media psyops campaign that beats it into everyone's heads that now we're at "Microsoft scale" it's ok for everything to "kinda suck", and if we're not careful... everything will.

Re: Supercharging the Git Commit Graph

#80

Earlier quoted context omitted.

Rebasing is fundamental to a proficient local workflow. Interactive rebasing with git rebase -p lets you craft and recraft your commits easily until they are readable and tell a story. The sooner a person can rebase, the sooner they can use Git. It’s not really possibly to proficiently or pleasantly use Git without rebasing.

I very strongly disagree with that. I'd rather someone be proficient with the staging index, and maybe even git stash, than rebasing. Maybe, if they are feeling fancy, `git add --interactive`, `git add --patch`, and/or `git commit --amend`. Rebasing is a fascinating footgun. I'd rather a messy story that includes details of how someone screwed up, then fixed their mistakes, than an entire branch I need to cherry pick…

Take some time and play around with these commands and read about them. They’re great... there’s nothing “fancy” or “advanced” about them. They just help you express your thoughts with Git more and better than if you don’t use them. Trust me, no one wants to read your free form Git history.
Post reply on HN