11–20 of 64 posts
GitHub really embraced the Microsoft-esque NIH with LFS, instead of adopting git-annex.
To its absolute detriment Here is a talk by a person who adores it: Yann Büchau: Staying in Control of your Scientific Data with Git Annex https://www.youtube.com/watch?v=IdRUsn-zB2s
Earlier quoted context omitted.
To its absolute detriment Here is a talk by a person who adores it: Yann Büchau: Staying in Control of your Scientific Data with Git Annex https://www.youtube.com/watch?v=IdRUsn-zB2s
While Yann has built many things with git-annex, we should be clear that the creator of git-annex is relatively singular, Joey Hess.
And an interview When power is low, I often hack in the evenings by lantern light. https://usesthis.com/interviews/joey.hess/
I tried using it for syncing large files in a collaborative repository, and the use of "magic" branches didn't seem to scale well.
git-annex has some really awkward documentation. You can apparently do, sort of, but not really, the same thing git-fetch-file[1] does, with git-annex: git fetch-file add https://github.com/icculus/physfs.git "**" lib/physfs-main git fetch-file pull `add` creates this at `.git-remote-files`: [file "**"] commit = 9d18d36b5a5207b72f473f05e1b2834e347d8144 target = lib/physfs-main repository = https://github.com/icculus/…
I use git-annex to manage all my data on all my drives. It automatically keeps track of which files are on which drives, it ensures that there are enough copies and it checksums everything. It works perfectly with offline drives. git-annex can be a bit hard to grasp, so I suggest to create a throw-away repository, following the walkthrough[1] and try things out. See also workflows[2]. [1] https://git-annex.branchable…
I wonder a bit whether that is ZFS, or git-annex, or maybe my disk, or sth else.
I use git-annex to manage all my data on all my drives. It automatically keeps track of which files are on which drives, it ensures that there are enough copies and it checksums everything. It works perfectly with offline drives. git-annex can be a bit hard to grasp, so I suggest to create a throw-away repository, following the walkthrough[1] and try things out. See also workflows[2]. [1] https://git-annex.branchable…
How much data do you have? I'm using git-annex on my photos, and that are around 100k-1M files, several TB of data, on a ZFS. In the beginning, everything was fine, but it starts to become increasingly slow, such that every operation takes several minutes (5-30 mins or so). I wonder a bit whether that is ZFS, or git-annex, or maybe my disk, or sth else.
Earlier quoted context omitted.
How much data do you have? I'm using git-annex on my photos, and that are around 100k-1M files, several TB of data, on a ZFS. In the beginning, everything was fine, but it starts to become increasingly slow, such that every operation takes several minutes (5-30 mins or so). I wonder a bit whether that is ZFS, or git-annex, or maybe my disk, or sth else.
It would be great to have comprehensive benchmarks for git lsf, git annex, dvc and alike. I am also always getting annoyed with one or the other , e.g. due to the hashing overhead, etc. However, in many cases the annoyances come with bad filesystem integration on Windows in my case.