Sustaining git-annex development • Hacker News Board

Re: Sustaining git-annex development

#31

Post by rsync » Mon, Jul 15, 2013, 1:52 PM UTC

zoul wrote: post #30

mjn wrote: post #25

Earlier quoted context omitted.

Wow, that looks great. With the discount it matches S3's storage pricing, but without bandwidth charges, with a more convenient interface (an ssh-reachable regular filesystem), and full-service support. Plus an interesting attempt at a "warrant canary": http://www.rsync.net/resources/notices/canary.txt

That canary thing is an interesting trick, previous discussion here: https://news.ycombinator.com/item?id=702247 . (Four years ago!)

We've been operating the warrant canary since early 2006.

Re: Sustaining git-annex development

#32

Post by voltagex_ » Mon, Jul 15, 2013, 2:27 PM UTC

rsync wrote: post #14

We (rsync.net) are supporting git-annex by offering all git-annex users a heavily discounted rsync.net account. This was announced a few days ago, but not on HN in any way: http://rsync.net/products/git-annex-pricing.html We've been explicitly supporting git-annex on our platform since our friend Jason Scott first showed it to us[1] and we will be contributing to the new campaign. [1] http://ascii.textfiles.com/archi…

In the interests of full transparency, I've been looking for alternatives to rsync.net. I haven't found any - although tarsnap comes very close.

I have an existing account, how can I move to this?

Re: Sustaining git-annex development

#33

Post by gnuvince » Mon, Jul 15, 2013, 3:50 PM UTC

Can anyone give me an example of their usage of git-annex? The project looks interesting, but the most immediate usage I see would be to add big files (e.g. a database dump, PDF documentation, etc.) to a git repository. What else do people use it for?

Re: Sustaining git-annex development

#34

Post by warp » Mon, Jul 15, 2013, 9:40 PM UTC

gnuvince wrote: post #33

Can anyone give me an example of their usage of git-annex? The project looks interesting, but the most immediate usage I see would be to add big files (e.g. a database dump, PDF documentation, etc.) to a git repository. What else do people use it for?

I use it to manage my "archive" folder, which contains large binary files which rarely change. This includes music, some tv series, operating system .iso images, backups of retired computers, etc..

The great thing about git annex is that each clone of the repository has the entire tree structure of the repository, but by default has none of the data. So if I'm going on a trip I can just cd into the right folder on my laptop and type "git annex get ." or "git annex get Windows*.iso". Being able to tab-complete all the files in the annex even though you don't have a copy of most of them makes it very convenient.

The numcopies constraints also help enforce redundancy on the data. I could use raid to have local redundancy, but that only protects against a harddisk crashing. If I have four repositories in different physical locations and set numcopies to three, then git annex helps make sure there are always enough copies of a file (in three different physical locations), so I won't lose data even if my house burns down.

Re: Sustaining git-annex development

#35

Post by semanticist » Tue, Jul 16, 2013, 10:47 PM UTC

asdfs wrote: post #13

SoftwareMaven wrote: post #12

Earlier quoted context omitted.

Time Machine to a local drive for immediate restores. Backblaze for disaster recovery. Time Machine could easily be replaced by cron and tar or some other local backup option. Other backup providers are easily found. This is not to say anything about git-annex's usefulness or the value of supporting the developer. It just surprises me when engineers are still struggling with backups. I can kind of understand "regular…

Can you use Time Machine to shuttle data via a USB stick? How well does it work when you have more than two drives that you want to keep mostly synchronized?

Time Machine does the one thing it does really well, which is incremental, local (or network-local) backups and easy restores.

It's not for snapshotting or mirroring. I used to use CarbonCopyCloner to clone my boot drive to an external USB drive every morning at ~4am, so if the internal drive on my iMac died I had an immediate bootable replacement.

(And when the internal drive DID die, I just rebooted the machine and took nearly a year to get around to replacing the internal drive!)