81–90 of 167 posts
A GB doesn't get you very far if you are working with raw audio and video.
Does it make sense to think about storing virtual machines images (.vmdk) in git on GitHub with LFS?
It's interesting that this uses smudge/clean filters. When I considered using those for git-annex, I noticed that the smudge and clean filters both had to consume the entire content of the file from stdin. Which means that eg, git status will need to feed all the large files in your work tree into git-lfs's smudge filter. I'm interested to see how this scales. My feeling when I looked at it was that it was not suffic…
Also, is there any reason Git LFS can't be used as a special remote for git-annex? It would provide an easy way for people to host their git-annex repos entirely on GitHub.
lfs-get SHA256 > file
lfs-store SHA256 Earlier quoted context omitted.
> (Not a git-annex user here) You could at least read the examples on the git-annex page[1] before passing judgement that the use cases are at all different (they're not). Instead of using a new 'lfs' command that ties you to GitHub, you use an 'annex' command (along with a few others). Git-annex does just fine "keeping track of larger objects inside your git project efficiently", and is no more divorced from your no…
I did read the use cases and the main pitch. I guess these don't highlight very well how it actually functions. Can I do "git add large.mp4 && git commit" the way I do now?
The documentation lays out the workflow: https://help.github.com/articles/configuring-large-file-stor...
As does the website: https://git-lfs.github.com/ (see: "Getting Started")
So does this mean the large files are actually versioned?
Earlier quoted context omitted.
You basically trade the ability to have diffs (nearly meaningless on binary files anyway) for representing large files as their SHA-256 equivalent values on a remote server. That's exactly what git-annex does. Except it can host on your own servers, or S3, or Tahoe-LAFS, or rsync.net, etc. And it's free software. And it supports multiple servers for the same repo, so you have redundancy. Adding an S3 remote is just s…
Indeed similar to git-annex, why does GitHub not support that? With GitLab we added support for that recently and it was pretty easy to do. Anyway, interesting that linking to sha's is becoming the default solution. Looking forward to playing with this and comparing the solutions.
Compare "Git Large File Storage"'s file spec:
version https://git-lfs.github.com/spec/v1
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
size 12345
bigstore
sha256
96e31e44688cee1b0a56922aff173f7fd900440f
Congrats to GitHub, but this leaves a sour taste in my mouth. FWIW, contributions are still welcome! And I hope there is still a future for bigstore.
Does this mean gamedevs might start droping Perforce for this? If its not too expensive maybe?
Most developers I know dropped Perforce for git and svn a long time ago. Configure them with the appropriate ignores and it works fine.
It's interesting that this uses smudge/clean filters. When I considered using those for git-annex, I noticed that the smudge and clean filters both had to consume the entire content of the file from stdin. Which means that eg, git status will need to feed all the large files in your work tree into git-lfs's smudge filter. I'm interested to see how this scales. My feeling when I looked at it was that it was not suffic…
As for git-lfs relative to git-fat: (1) the Go implementation is probably sensible because Python startup time is very slow, (2) git-lfs needs server-side support so administration and security is more complicated, (3) git-lfs appears to be quite opinionated about when files are transferred and inflated in the working tree. The last point may severely limit ability to work offline/on slow networks and may cause interactive response time to be unacceptable. Some details of the implementation are different and I'd be curious to see performance comparisons among all of our tools.
Has someone had a closer look and can say how this compares to Git-Annex?
This and git-annex (and git-fat and others) use the same basic architecture of storing links in Git and schlepping the binaries around separately. Git-annex renames binaries with their SHA256 hashes, puts them in a .git/annex/ dir, and replaces files in the working dir with symlinks. Git-LFS seems to use small metadata pointer files (SHA256 hash, file size, git-lfs version) instead of symlinks. Not sure whether the f…
I hesitate to say this means git-lfs is not distributed at all, but it seems significantly less distributed than git-annex, which can keep track of files that might be in Glacier, or on an offline drive, or a repo cloned on a nearby computer, and so can be used in a more peer-to-peer fashion when storing and retrieving the large files.