Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Git Exercises

gitexercises.fracz.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Git Exercises

#11

Post by [deleted] »

rubyist5eva wrote: post #8
notafraudster wrote: post #6

Responding in hopes some Git non-novices are here and can give some quick advice. I have a fairly large Git repo with 5 years of commits from numerous team members including a bunch of non-technical people who had never used Git before. There were two major issues: 1. We started off storing binary files -- mostly images, but also a ton of raw data files that got versioned every day or two -- in this repro and it spir…

You should use git-lfs for these files. Probably some wizardry with `filter-branch` and adding these files to lfs could debloat your repo once you do a fresh clone from that.

[deleted]

Re: Git Exercises

#12

Post by btschaegg »

notafraudster wrote: post #6

Responding in hopes some Git non-novices are here and can give some quick advice. I have a fairly large Git repo with 5 years of commits from numerous team members including a bunch of non-technical people who had never used Git before. There were two major issues: 1. We started off storing binary files -- mostly images, but also a ton of raw data files that got versioned every day or two -- in this repro and it spir…

On 1.:

As others already noted, Git has a GC-mechanism, which means that objects can still linger around in any copy of your repo for a while. And if you need to version binary files, you'd better use git-lfs or git-annex. Obviously, if you don't need them, just nuking them outright with BFG or `git filter-branch` is fine, too.

If you'd like to try git-lfs: It also includes tooling to retroactively migrate your Repository[1], but that'll re-write your history (although BFG obviously does that, too).

If GCing your repo will not reduce its size, you'll probably have to hunt down any remnant branch and/or tag that might reference the old history and thus "keep it alive".

On 2.:

I'm not entirely certain I understand you correctly. Which commits were duplicated? The resulting merge commits?

Assuming that, if you wanted to, you could probably build some shell script to get rid of them (or use something like `git checkout prod; git rebase -ir ` and remove the duplicated merges yourself).

But from a repo perspective, this shouldn't cause too much trouble (i.e. the additional space required will be negligible), and doing so would, again, mean that you'll rewrite history, potentially causing issues for others who still have local copies referencing your old commits.

Also: If you try to go the rebase route: Make sure you understand the log Git will create for you. Using `-r`, it will preserve merges, but how they are represented is not very intuitive and you'll have to wrap your head around that first.

You could also try to achieve the same result with `git filter-branch` and `--commit-filter`. In this case, you'd probably want to write a script that only performs the `git commit-tree` command if the tree ID passed to the filter is not the same tree as the first parent commit was referencing already (this should weed out all commits that don't change anything).

[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/man/git-...

Edit: As an idea for 2.:

Describing issues in commit histories as prose text is tricky :). If your problem looks different than I assumed, you could try to create a bogus Git repo that showcases the pattern in its history and put it on GitHub as a reference.

You can create "empty" commits for that via `git commit --allow-empty`.

Re: Git Exercises

#14

Post by yissp »

mcstafford wrote: post #13

> git start master I've been using git for years, and can't find documentation for start. https://gitexercises.fracz.com/exercise/master https://git-scm.com/docs

The configure.sh script you run at the start creates a few aliases (start, verify, exercises). See https://raw.githubusercontent.com/fracz/git-exercises/master...

Re: Git Exercises

#15

Post by notafraudster »

btschaegg wrote: post #12
notafraudster wrote: post #6

Responding in hopes some Git non-novices are here and can give some quick advice. I have a fairly large Git repo with 5 years of commits from numerous team members including a bunch of non-technical people who had never used Git before. There were two major issues: 1. We started off storing binary files -- mostly images, but also a ton of raw data files that got versioned every day or two -- in this repro and it spir…

On 1.: As others already noted, Git has a GC-mechanism, which means that objects can still linger around in any copy of your repo for a while. And if you need to version binary files, you'd better use git-lfs or git-annex. Obviously, if you don't need them, just nuking them outright with BFG or `git filter-branch` is fine, too. If you'd like to try git-lfs: It also includes tooling to retroactively migrate your Repos…

RE #2:

(First, thanks for everything!)

Our regular routine at the time: Imagine production and dev are even. We add 5 regular commits to dev. We PR merge dev -> production. We do not do squash and merge. We do not delete dev. Dev and master are now even but master has the merge commit as well.

Flash forward several years. Looking at history, we now see each of those 5 commits for a run of about a year appear multiple times (at the peak, 5 times). The exact same commit, same commit message, etc. This is supposed to be impossible with git and we don't know what caused it, but assume that it was a user error either trying to resolve #1 or just by people who didn't know how to merge merging. Whatever error caused it clearly occurred a few times because for a while the commits appear 5 times, then 4, then 3, then 2, then back to normal, so I suspect whatever the cause, it happened 4 times, duplicating a range of commits each time.

We'd like to go back and de-duplicate, keeping the whole history that led us to now but only having 1 of every commit. I don't fully care about branch history. I just care about blame and about commit counts for all the people who worked on the project. I am fine with the commit counts falling with the duplicate commits removed, but just not falling to 0 like it would if we started from scratch.

We are not worried about local copies (it's a collaborators-only repo that there'd be no reason for uninvolved people to fork) and right now we have no collaborators since the project is grant funded and we don't have an active grant.

That's half the reason I'm trying to clean up these nuisance issues now before other people are involved again.

Re: Git Exercises

#16

Post by notafraudster »

auxym wrote: post #7
notafraudster wrote: post #6

Responding in hopes some Git non-novices are here and can give some quick advice. I have a fairly large Git repo with 5 years of commits from numerous team members including a bunch of non-technical people who had never used Git before. There were two major issues: 1. We started off storing binary files -- mostly images, but also a ton of raw data files that got versioned every day or two -- in this repro and it spir…

1. If the large objects are not referenced by any commit anymore, they should automatically get garbage collected eventually, but you can force this process with git gc: https://git-scm.com/docs/git-gc 2. Never seen that one. There's probably an arcane filter-branch command that could detect the duplicates and squash them, but personally I'd just leave it alone. How often do you look at many-years old history anyways…

Thanks. I'll read through the gc docs. If GCing doesn't help then at least that helps me situate further searches better. Appreciate it.

Re: Git Exercises

#17

Post by cgriswald »

mcstafford wrote: post #13

> git start master I've been using git for years, and can't find documentation for start. https://gitexercises.fracz.com/exercise/master https://git-scm.com/docs

From the site:

> And the git start?

> That is the first alias configured by the script described above. It initializes the first exercise that is on master branch. Read the instructions and solve it!

Re: Git Exercises

#18

Post by btschaegg »

btschaegg wrote: post #12
notafraudster wrote: post #6

Responding in hopes some Git non-novices are here and can give some quick advice. I have a fairly large Git repo with 5 years of commits from numerous team members including a bunch of non-technical people who had never used Git before. There were two major issues: 1. We started off storing binary files -- mostly images, but also a ton of raw data files that got versioned every day or two -- in this repro and it spir…

On 1.: As others already noted, Git has a GC-mechanism, which means that objects can still linger around in any copy of your repo for a while. And if you need to version binary files, you'd better use git-lfs or git-annex. Obviously, if you don't need them, just nuking them outright with BFG or `git filter-branch` is fine, too. If you'd like to try git-lfs: It also includes tooling to retroactively migrate your Repos…

Something I just thought of:

If you've GCd your repo and you're sure there are no references to old commits laying around (also keep in mind remote branches and so on!), this might help you discover large objects that are still in your "new" history:

If run in bash, this should print the 20 largest objects in your repo and their size (in bytes):

  git rev-list --all \
  | xargs -n1 git ls-tree -r \
  | awk '$2 == "blob" { print $3 }' \
  | sort -u \
  | while IFS= read blob; do
      echo "$blob $(git cat-file blob $blob | wc -c)";
  done \
  | sort -rnk2,2 \
  | head -20
If you find some blob that is too large, you could then search for its name like this:

  large_blob=

  git rev-list --all \
  | xargs -n1 git ls-tree -r \
  | fgrep $large_blob

Re: Git Exercises

#19

Post by dengsauve »

yissp wrote: post #14
mcstafford wrote: post #13

> git start master I've been using git for years, and can't find documentation for start. https://gitexercises.fracz.com/exercise/master https://git-scm.com/docs

The configure.sh script you run at the start creates a few aliases (start, verify, exercises). See https://raw.githubusercontent.com/fracz/git-exercises/master...

Thank you for posting this, when I cloned the repo initially from the site, configure.sh was not included.

Re: Git Exercises

#20

Post by justusthane »

antoniuschan99 wrote: post #3
mcadenhe wrote: post #2

Excited to give this a try and see if it will help me expand my day-to-day git-fu. I like interactive learning resources like this. A similar one was posted to hn a while back for postgres and going through it taught me a lot. https://pgexercises.com/

Thanks for this :). A few more: https://jskatas.org https://www.flexboxfroggy.com https://www.cssgridgarden.com

https://vim-adventures.com/

https://www.vimgolf.com/

Post reply on HN