Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artisanal handcrafted Git repositories

drew.silcock.dev

Post reply on HN
Print view View original

31–40 of 74 posts

Re: Artisanal handcrafted Git repositories

#31

Post by bradfitz »

My recent horror from some git work was discovering how git sorts its tree objects.

The docs just say to sort by C locale (byte-order sorting). Easy. Except git was sometimes rejecting my packfiles as being bogus per its fsck code, saying my trees were misordered.

TURNS OUT THERE'S AN UNDOCUMENTED RULE: you need to append an implicit forward slash to directory tree entry names before you sort them.

That forward slash is not encoded in the tree object, nor is the type of the entry. You just put the 20 byte SHA1 hash, which is to either a blob or a hash (or a commit for submodules).

So you can have one directory with directory "testing" and file "testing.md" and it'll sort differently than a directory with two files "testing" and "testing.md".

You can see a repro at https://gist.github.com/bradfitz/4751c58b07b57ff303cbfec3e39...

(So to verify whether a tree object is formatted correctly, you need to have the blobs of all the entries in the tree, at least one level)

Re: Artisanal handcrafted Git repositories

#32

Post by steveklabnik »

chubot wrote: post #25
shivasaxena wrote: post #13

Earlier quoted context omitted.

I agree, but only in theory. Projects like gitoxide have been in development for years now.

Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2 I think Github and most big Git hosts use it

libgit2 has a ton of compatibility issues, especially around authentication, that make it only useful in some circumstances.

(gitoxide is a similar project but in Rust, it's not ready for the big time either, though it keeps on getting better!)

Re: Artisanal handcrafted Git repositories

#33

Post by 3eb7988a1663 »

chubot wrote: post #25
shivasaxena wrote: post #13

Earlier quoted context omitted.

I agree, but only in theory. Projects like gitoxide have been in development for years now.

Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2 I think Github and most big Git hosts use it

Jujitsu threw in the towel and is shelling out to the git CLI because of minor variations in libgit vs the binary.

Failing to find a write-up, but there was this lobster thread[0] where someone from GitLab reported they had to do the same owing to some discrepancies vs the binary -where all of the real development happens.

[0] https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released

Re: Artisanal handcrafted Git repositories

#34

Post by MrJohz »

fiddlerwoaroof wrote: post #18
shivasaxena wrote: post #13

Earlier quoted context omitted.

I agree, but only in theory. Projects like gitoxide have been in development for years now.

I wrote a nearly complete implementation of git file format parsers in Common Lisp over like a month of evenings and weekends. I’m sure there are hard parts between where I am and a full git implementation but you can get quite a bit of utility out of a relatively small amount of effort.

It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult.

I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues with SSH handling that they've since switched to directly invoking the Git binary for those operations.

Re: Artisanal handcrafted Git repositories

#35

Post by fiddlerwoaroof »

MrJohz wrote: post #34
fiddlerwoaroof wrote: post #18

Earlier quoted context omitted.

I wrote a nearly complete implementation of git file format parsers in Common Lisp over like a month of evenings and weekends. I’m sure there are hard parts between where I am and a full git implementation but you can get quite a bit of utility out of a relatively small amount of effort.

It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult. I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues wit…

There aren’t that many weird states a git repository can be in: the on-disk format of the repository is too simple for that. The hard part has to do with the various protocols for transferring objects around.

Re: Artisanal handcrafted Git repositories

#36

Post by Dylan16807 »

3eb7988a1663 wrote: post #33
chubot wrote: post #25

Earlier quoted context omitted.

Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2 I think Github and most big Git hosts use it

Jujitsu threw in the towel and is shelling out to the git CLI because of minor variations in libgit vs the binary. Failing to find a write-up, but there was this lobster thread[0] where someone from GitLab reported they had to do the same owing to some discrepancies vs the binary -where all of the real development happens. [0] https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released

But nothing in that description of problems is tied to the repository format.

Re: Artisanal handcrafted Git repositories

#39

Post by antonvs »

BobaFloutist wrote: post #24
derefr wrote: post #16

Earlier quoted context omitted.

FYI: the pinch-to-zoom gesture from mobile browsers (from before websites were mobile-responsive) has also long been implemented for all modern desktop browsers. It's viewport zoom, which is far better than the font-scaling zoom you get by pressing Cmd-+, and makes this site easily readable. (The much-less-well-known mobile double-tap-on-text gesture [it zooms-to-fit whatever element you tapped on to the width of the…

Double tap on text highlights it for me. Is that an Iphone/android thing or what?

On my iPad in Safari and Pixel Android phone in Firefox, one-finger double tap on text does the fit to viewport.

On my Ubuntu laptop in Chrome, I couldn’t find a way to make it work - even tapping the touchscreen didn’t work. But I’m not using the stock Ubuntu GUI, so it could be that (LXqt+XMonad).

Re: Artisanal handcrafted Git repositories

#40

Post by deathanatos »

fiddlerwoaroof wrote: post #35
MrJohz wrote: post #34

Earlier quoted context omitted.

It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult. I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues wit…

There aren’t that many weird states a git repository can be in: the on-disk format of the repository is too simple for that. The hard part has to do with the various protocols for transferring objects around.

I think there's more corners out there than most people would give credit to? Just off the top of my head: files in the index (but maybe this isn't "weird enough"), rebasing but paused, rebasing with conflicts, merge with conflicts, cherry-picking but conflicts, middle of a bisect with all the state that implies, alternate objects dirs, alternate working dirs, submodules and all of their weirdness, and a "bare" repo.

Heck, had my PS1 return an error this week after I created a separate working dir for a repo and cd'd into it. Did you know .git can be a normal file? I didn't when I wrote my PS1.

Post reply on HN