Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On undoing, fixing, or removing commits in git

sethrobertson.github.io

Post reply on HN
Print view View original

61–70 of 74 posts

Re: On undoing, fixing, or removing commits in git

#61

Post by mtdewcmu »

pyre wrote: post #59
mtdewcmu wrote: post #58

Earlier quoted context omitted.

I've read a lot about git. The docs generally don't pick apart what's inside the .git directory.

- The history items are stored as commit objects that are identified as a SHA-1 sum of the contents (including meta-data like Authored By, Committed By, etc). - One of those meta-data items is "Parent Commit," so if you change one item in history, it changes the SHA-1 sum of all subsequent items (because at the very least they all need to be re-parented). - All of the commit objects are stored under .git/objects. - B…

It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. What are the SHA-1 sums of? Are they of the entire snapshot, or the delta? I went into objects/ and ran `sha1sum $objfile`, and the sum did not match the file name. So that remains obscure. `file $objfile` could not identify the format; it gave nonsense.

Thanks for the help.

>One of those meta-data items is "Parent Commit," so if you change one item in history, it changes the SHA-1 sum of all subsequent items (because at the very least they all need to be re-parented).

What sequence of operations would change a history item in that way?

Re: On undoing, fixing, or removing commits in git

#62

Post by pyre »

mtdewcmu wrote: post #61
pyre wrote: post #59

Earlier quoted context omitted.

- The history items are stored as commit objects that are identified as a SHA-1 sum of the contents (including meta-data like Authored By, Committed By, etc). - One of those meta-data items is "Parent Commit," so if you change one item in history, it changes the SHA-1 sum of all subsequent items (because at the very least they all need to be re-parented). - All of the commit objects are stored under .git/objects. - B…

It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. What are the SHA-1 sums of? Are they of the entire snapshot, or the delta? I went into objects/ and ran `sha1sum $objfile`, and the sum did not match the file name. So that remains obscure. `file $objfile` could not…

> It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible.

I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the git source code:

  user@host ~/src/git % cat .git/logs/HEAD
  0000000000000000000000000000000000000000 d7aced95cd681b761468635f8d2a8b82d7ed26fd First Last  1387237920 -0500	clone: from https://github.com/git/git.git

  user@host ~/src/git % git reflog
  d7aced9 HEAD@{0}: clone: from https://github.com/git/git.git
Note: `HEAD` is a reference to the current branch. E.g.:

  ~/src/git $ cat .git/HEAD
  ref: refs/heads/master

  ~/src/git $ cat .git/refs/heads/master
  d7aced95cd681b761468635f8d2a8b82d7ed26fd
It's also of note that branches are referred to as 'references' too, hence storing them under `.git/refs/`.

> What are the SHA-1 sums of? Are they of the entire snapshot, or the delta? I went into objects/ and ran `sha1sum $objfile`, and the sum did not match the file name. So that remains obscure.

See: http://stackoverflow.com/questions/5290444/why-does-git-hash...

[1]: Since the local repository was created. This information does not sync between local and remote.

Re: On undoing, fixing, or removing commits in git

#63

Post by gemma »

crystaln wrote: post #20
gemma wrote: post #13

Earlier quoted context omitted.

No, that advice from the article is fundamentally broken. Outside of the garbage collection system (which runs by default after what, 30 days? 90?), Git doesn't delete committed content. Any commit you "lose" through rebasing, amending, resetting, etc. can always be recovered. It's a little more complicated than renaming a directory, sure, but it's important, and it's not something a Git tutorial should ignore. Git I…

I'm not 100% sure this is true, however it is also a fundamental flaw of git. There should be a way to remove commits permanently in order to remove mistakenly checked in large files or private content. It's also definitely not true with uncommitted changes, including gitignored files.

I still don't see the "fundamental flaw". Non-reachable commits are automatically deleted by the garbage collection system, which can be also be run manually. Accidental commits with large files or private content can be "modified" (technically copied and rewritten, since individual commits are immutable) with rebase, amend, filter-branch, etc. Those operations make the original commits unreachable, so garbage collection takes care of deleting them.

And like I already said above, data loss can occur when you're working with uncommitted changes, just like in most other version control systems. If the content is not under version control (in this case, not in a git commit), it's not safe.

Honestly, you guys should go watch Linus Torvalds' presentation at Google about Git. The entire point, the massive problem he was trying to solve, was preservation and verification of data integrity.

Re: On undoing, fixing, or removing commits in git

#64

Post by mtdewcmu »

pyre wrote: post #62
mtdewcmu wrote: post #61

Earlier quoted context omitted.

It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. What are the SHA-1 sums of? Are they of the entire snapshot, or the delta? I went into objects/ and ran `sha1sum $objfile`, and the sum did not match the file name. So that remains obscure. `file $objfile` could not…

> It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the…

>I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the git source code:

I think it's more or less the DAG represented as an adjacency list. I'd have to think a bit about why there is a separate log file for each branch. It seems that there's some redundancy in doing that, and I'm wondering what the advantages and disadvantages are of splitting the history up in that way.

>It's also of note that branches are referred to as 'references' too, hence storing them under `.git/refs/`.

I've developed a loathing of excessive hierarchies/trees, so I'd rather see them flattened in a single directory. But that makes sense.

>See: http://stackoverflow.com/questions/5290444/why-does-git-hash....

That's a good link. What's in an object? If an object corresponds to a commit, then it must aggregate data about changes to multiple files.

Re: On undoing, fixing, or removing commits in git

#65

Post by eru »

mtdewcmu wrote: post #39
mikeash wrote: post #9

"Strongly consider taking a backup of your current working directory and .git to avoid any possibility of losing data as a result of the use or misuse of these instructions." WTF? What is the point of a version control system if you have to take backups of it to avoid losing data when performing certain operations? I use git, I like git, but certain aspects of it are fundamentally broken.

I agree that git is both a great advance and seems fundamentally broken at the same time. One of git's advances is that it treats commits as snapshots of the entire tree rather than diffs[1]. A snapshot might as well be a tarball of the whole directory, except that git uses references to previous snapshots to store it efficiently. So in this aspect, git is like a backup tool plus compression. It's not quite a useful…

> I'm thinking that the repository could be moved out of the working directory and placed in its own file that's not invisible.

Symlinks are your friends.

> then repos would be portable and you could ftp them.

tar might come in handy.

> The backup functionality could be separated from the history-tracking functionality, so you could make backups freely without adding noise to the commit history. A backup would basically be a tarball that you could append to a repo file, taking advantage of previous entries for compression.

You can already do this. You can have commits without ancestors or descendants in your repository, and they will still benefit from delta compression.

Re: On undoing, fixing, or removing commits in git

#66

Post by mtdewcmu »

pyre wrote: post #62
mtdewcmu wrote: post #61

Earlier quoted context omitted.

It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. What are the SHA-1 sums of? Are they of the entire snapshot, or the delta? I went into objects/ and ran `sha1sum $objfile`, and the sum did not match the file name. So that remains obscure. `file $objfile` could not…

> It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the…

I'm guessing that the reason each branch has its own history is probably related to the goal of only appending new entries at the end of things. Since any branch can be under development, they need their own files. It sort of makes sense.

Re: On undoing, fixing, or removing commits in git

#67

Post by mtdewcmu »

eru wrote: post #65
mtdewcmu wrote: post #39

Earlier quoted context omitted.

I agree that git is both a great advance and seems fundamentally broken at the same time. One of git's advances is that it treats commits as snapshots of the entire tree rather than diffs[1]. A snapshot might as well be a tarball of the whole directory, except that git uses references to previous snapshots to store it efficiently. So in this aspect, git is like a backup tool plus compression. It's not quite a useful…

> I'm thinking that the repository could be moved out of the working directory and placed in its own file that's not invisible. Symlinks are your friends. > then repos would be portable and you could ftp them. tar might come in handy. > The backup functionality could be separated from the history-tracking functionality, so you could make backups freely without adding noise to the commit history. A backup would basica…

The idea is to make it less monolithic. I'm digging more into the inner workings of git now. The functionality might already be in there, just not obvious.

Re: On undoing, fixing, or removing commits in git

#68

Post by pyre »

mtdewcmu wrote: post #64
pyre wrote: post #62

Earlier quoted context omitted.

> It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the…

>I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the git source code: I think it's more or less the DAG represented as an adjacency list. I'd have to think a bit about why there is a separate log file for each branch. It seems that there's some redundancy i…

> I think it's more or less the DAG represented as an adjacency list. I'd have to think a bit about why there is a separate log file for each branch. It seems that there's some redundancy in doing that, and I'm wondering what the advantages and disadvantages are of splitting the history up in that way.

Think of each branch as a pointer. Then realize that you can make that pointer point anywhere on the DAG, even to parts of the DAG that have no connection to each other. The `reflog` is a (local, non-comprehensive) history of where that pointer has pointed. That's why there is a separate log for each branch. I guess that technically they could have a single log file and add another field to specify the branch, but using the same directory tree structure as under .git/refs/ makes the mental model simpler (and probably a performance improvement not to have to parse the reflog for every branch just to see the reflog for one branch).

> I've developed a loathing of excessive hierarchies/trees, so I'd rather see them flattened in a single directory. But that makes sense.

I'm not sure what branches living under .git/refs has to do with excessive hierarchies/trees. There are enough things stored in the .git directory, that if you mashed them all together it wouldn't make any sense.

> What's in an object?

If you really care to dive deeper, you can check objects here: https://github.com/git/git/blob/master/object.h

You can get a shorter version towards the bottom of the git manpage (e.g. `man git`):

  IDENTIFIER TERMINOLOGY
         
             Indicates the object name for any
             type of object.
  
         
             Indicates a blob object name.
  
         
             Indicates a tree object name.
  
         
             Indicates a commit object name.
  
         
             Indicates a tree, commit or tag
             object name. A command that takes a
              argument ultimately wants
             to operate on a  object but
             automatically dereferences 
             and  objects that point at a
             .
  
         
             Indicates a commit or tag object
             name. A command that takes a
              argument ultimately
             wants to operate on a  object
             but automatically dereferences 
             objects that point at a .
  
         
             Indicates that an object type is
             required. Currently one of: blob,
             tree, commit, or tag.
  
         
             Indicates a filename - almost always
             relative to the root of the tree
             structure GIT_INDEX_FILE describes.

Re: On undoing, fixing, or removing commits in git

#69

Post by pyre »

mtdewcmu wrote: post #66
pyre wrote: post #62

Earlier quoted context omitted.

> It looks like .git/logs contains the history. It looks like the file format is a space-separated list, with the format "$parentcommitsha1 $newcommitsha1 ... $commitmessage". That's fairly comprehensible. I've never looked at .git/logs, but it looks like that is used by the `git reflog` command. It's basically a history (or log) of every commit that a particular reference has pointed to[1]. For example, I cloned the…

I'm guessing that the reason each branch has its own history is probably related to the goal of only appending new entries at the end of things. Since any branch can be under development, they need their own files. It sort of makes sense.

I still think that you're a little confused. The reflog is a "history of where this branch has pointed since the repository was created/cloned." If I clone a repository with a history of 100 commits on the 'master' branch, the reflog for the 'master' branch will only have one entry. You can completely delete the `.git/logs` and still run `git log` successfully.

Here's an example:

  $ git clone blah
  
  DAG:
  
    A - B - C - D - E
        \
         Z - X - Y
  
  
  Branches:
  
   master => E
   topic/new-feature => Y
  
  
  reflog:
  
    master
      E - clone from blah
  
    topic/new-feature
      Y - clone from blah
Notice how cloning a repository with an existing DAG doesn't populate the reflog. It just give it a single entry saying that the branch was updated from 'nothing' to whatever commit it was pointing to remotely.

Now let's change where 'master' is pointing:

  $ git reset master C
  
  
  DAG:
  
    A - B - C - D - E
        \
         Z - X - Y
  
  
  Branches:
  
   master => C
   topic/new-feature => Y
  
  
  reflog:
  
    master
      E - clone from blah
      C - reset to C
  
    topic/new-feature
      Y - clone from blah
Notice how the reflog is a history of the values that the branch was referencing, but is not the history as what you get when you run 'git log'. After the reset, 'git log master' would show you commits A, B and C, but A and B are nowhere in the reflog.

Re: On undoing, fixing, or removing commits in git

#70

Post by mtdewcmu »

pyre wrote: post #69
mtdewcmu wrote: post #66

Earlier quoted context omitted.

I'm guessing that the reason each branch has its own history is probably related to the goal of only appending new entries at the end of things. Since any branch can be under development, they need their own files. It sort of makes sense.

I still think that you're a little confused. The reflog is a "history of where this branch has pointed since the repository was created/cloned." If I clone a repository with a history of 100 commits on the 'master' branch, the reflog for the 'master' branch will only have one entry. You can completely delete the `.git/logs` and still run `git log` successfully. Here's an example: $ git clone blah DAG: A - B - C - D -…

I see. I started reading the internals chapter at[1]. This free book seems better than the O'Reilly book, which I bought.

So the DAG is actually stored inside objects. The contents of the objects directory could be described by a relational schema, and I think that would make it easier for a lot of people to understand (myself included):

  Blob
  - sha1hash (primary key)
  - contents (blob)

  Tree
  - sha1hash (primary key)

  TreeEntry
  - treeid (foreign key into Tree)
  - mode (mode of blob/subtree)
  - type ("blob" or "tree")
  - objectid (foreign key into Tree or Blob)
  - name

  Commit
  - sha1hash (primary key)
  - tree (foreign key into Tree)
  - parent (foreign key into Commit)
  - author
  - committer
  - comment
The tree entries are actually denormalized and stored as a list inside the tree. You could represent this more accurately with XML. But who likes XML?

[1] http://git-scm.com/book

Post reply on HN