Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you store photos and videos?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 104 posts

Re: Ask HN: How do you store photos and videos?

#41

Post by xte »

xte wrote: post #32

Hum, let me answer per points: 0. I may empathize BUT if you value data on them... Well you ask something like how I can have my wife drunk and my barrel full (ancient Italian proverb). Sometimes you can "optimize" your storage for instance lowering a bit quality of images that you may do not need at super-high resolution, stripping metatada, change formats, ... Depending on the case you may save big size of storage.…

Ps

  to answerless downvoters: try to explain why. Maybe you have really interesting reasons so sharing them help others, me included, to improve a bit. Or maybe you have no reasons so others are more likely to get interesting answers.

  I know that use their own fingertips on a keyboard, with brain connected, might be something tiresome but normally turn out to be helpful. Sometimes even to the writer itself that realize some new things in the process of explaining them to others.

Re: Ask HN: How do you store photos and videos?

#42

Post by ahnick »

andrewstuart2 wrote: post #21
sokoloff wrote: post #12

I have the primary store on a RAID-Z2 zfs volume on a FreeNAS server in the basement. That backs up all photos to Amazon Photos ("free" with Amazon Prime, which I have anyway) and important things (family photos, family videos, and legal docs) to S3 with lifecycle policies defined. Family photos are additionally rsynced nightly [one-way] to a Synology server (just because I had it before I built the FreeNAS storage a…

You might want to look into duplicity [1] for client-side encryption and as a general incremental backup solution. I've been using it for a while, and had to restore a couple times, and it's both pretty intuitive and works as expected. It's essentially just a wrapper on GnuPG and rsync/tar with a bunch of storage backends. [1] http://duplicity.nongnu.org/

If you like duplicity you might prefer duplicacy [1]. I've found it's incremental backup solution to be far superior. The only negative might be the licensing, but with the free version you have access to all the code for non-commercial use and that suits my personal needs just fine.

[1] https://github.com/gilbertchen/duplicacy

Re: Ask HN: How do you store photos and videos?

#43

Post by apaprocki »

larrysalibra wrote: post #5

I use iCloud Photo Library which solves most of your problems except client side encryption - which I agree is a big negative in its corner. It works very well if you're all in in the Apple ecosystem. https://support.apple.com/en-us/HT204264

To more specifically answer OP's questions.. I'm a fellow Arq user. On your mobile devices / laptop(s) you would configure iCloud Photo Library to optimize automatically so that it frees up all the disk space. On a stationary Mac of any flavor, configure it to store the entire repository locally and then run Arq on that computer to back up the entire archive to N providers or your own off-site storage. This does have the slight drawback of requiring a macOS device in order to have the full photo library and run Arq, but paired with the usability of accessing any of your photos on any device securely, you basically get everything you will ever need. This can be something "cheaper" like a used stationary laptop with external SSD attached if necessary, or you can just park a Mac Mini somewhere if you can afford it. Once it's set up, you'll never need to touch a thing and everything will be encrypted before being backed up. (Important distinction: iCloud Photo Library is not backup and explicitly says so.. it is merely the front-end UI for quickly accessing any photo on any device.)

Re: Ask HN: How do you store photos and videos?

#44

Post by ptoomey3 »

larrysalibra wrote: post #5

I use iCloud Photo Library which solves most of your problems except client side encryption - which I agree is a big negative in its corner. It works very well if you're all in in the Apple ecosystem. https://support.apple.com/en-us/HT204264

While I voted for google photos in https://news.ycombinator.com/item?id=18770681, I will say that apple photos is nice if you have zero need to share a library with family members. For better or worse, apple Unlike third-party uploaders, Apple photos can “cheat” and upload no matter what. That by itself makes it the most reliable for Apple ecosystem folks. That said, I’ve not had any issues in recent history with the google photos uploader.

Re: Ask HN: How do you store photos and videos?

#45

Post by ptoomey3 »

rasikjain wrote: post #35
ptoomey3 wrote: post #19

Earlier quoted context omitted.

This! I’ve tried every option under the sun at least twice and google photos wins. My current system has been this for the past 2 years or so: 1. Create a family gmail account (ex. house-yourlastname@gmail.com) 2. Install Google Photos on the both you and your spouse’s phone and set the automatic photo upload to upload to the house-yourlastname@gmail.com account. 3. Create a new account on your local home server (ex.…

I second the Google photo option as well. Before this, I tried storing and taking backups in external disk, This resulted in multiple copies. Switched to using google photos as primary backup mechanism. Working great so far. I also like their search feature, auto-create and reminder of previous year photos.

Also, while talking up google photos, I must note the home hub is worth the price of admission even if all you use it for is as a photo frame for google photos. I’m mostly an all apple house (HomeKit, etc). But, google photos plus the home hub has been a huge hit with the family.

Re: Ask HN: How do you store photos and videos?

#46

Post by subpixel »

ptoomey3 wrote: post #19
rococode wrote: post #15

Google Photos has been my goto for a while now. The search feature is great and I use it frequently. For example, I was recently looking for a blue screen error code I'd taken a picture of months ago, and was able to find it within seconds by searching "monitor". It's really incredibly good, I'd say whatever model is on the backend is probably close to state-of-the-art. I recently hit the free limit, but for ~$3 a mo…

This! I’ve tried every option under the sun at least twice and google photos wins. My current system has been this for the past 2 years or so: 1. Create a family gmail account (ex. house-yourlastname@gmail.com) 2. Install Google Photos on the both you and your spouse’s phone and set the automatic photo upload to upload to the house-yourlastname@gmail.com account. 3. Create a new account on your local home server (ex.…

Thanks, I may follow this to a 't'. One question: you mention 'your local home server' but it sounds like any Mac will do as long as the account is always logged in, no?

Re: Ask HN: How do you store photos and videos?

#47

Post by gumby »

What long term editable “master” format do you use for videos? Most of mine have become unviewable over the years as video formats changed and support faded away, except some that were edited and the edits exported into crappy mpegs. I can export now into the hopefully longer-lived h.264 but is that reasonable for source files?

This is both a container and codec problem which just makes life even more complicated.

Re: Ask HN: How do you store photos and videos?

#48

Post by LinuxBender »

xte wrote: post #36
LinuxBender wrote: post #27

I use sftp+lftp [1] (faster and more secure than rsync) to transfer them to one of my backup servers. That server uses rsnapshot to keep diffs and also backs up to multiple external USB drives. The same method of sftp+lftp can be used to back files up off-site. Sftp can't touch the snapshots which protects against malicious access (cryptolockers and such) [1] - https://tinyvpn.org/sftp/#lftp

Hem could you elaborate on "faster and more secure than rsync" rsync can use various kind of transport mechanism, you also may use other not directly supported by rsync in the middle simply piping though them. mbuffer, cryptcat, ncat, there are MANY options. Also in term of "safety" (or encryption) many kind of performance depend on how you encrypt your data, and how you compress your transfer because they are pretty…

Certainly. The lftp mirror subsystem allows you to break up batches of files, or even a single file using p-get into multiple streams. Rsync can't do that. Rsync also doesn't work within a chroot environment, whereas lftp does. The exception is if you use native rsync, which means you are not encrypting the stream. You then get chroot, but not privacy.

CPU overhead isn't really a limiting factor these days, assuming your computer is less than a decade old.

With sftp chroot, I can let people upload files without executing files, while preventing eavesdropping. rsync does not support TLS without complex stunnel wrappers on each side.

Re: Ask HN: How do you store photos and videos?

#49

Post by votepaunchy »

dawnerd wrote: post #28

Gsuite with unlimited data and google photos with original quality enabled. Photos are backed up once a month to aws in lower but still acceptable quality. (Might be changing that as storage costs are starting to get really brutal). Also generate quite a bit of video for a side project of mine and that also goes right to google drive in a team share. Don’t have the space or funds to store those elsewhere. Talking ~7t…

> storage costs are starting to get really brutal ... Talking ~7tb of video and premier pro project files.

AWS recently announced Glacier deep-archive at $1/TB-month (standard Glacier being $4/TB-month).

Re: Ask HN: How do you store photos and videos?

#50

Post by xte »

LinuxBender wrote: post #48
xte wrote: post #36

Earlier quoted context omitted.

Hem could you elaborate on "faster and more secure than rsync" rsync can use various kind of transport mechanism, you also may use other not directly supported by rsync in the middle simply piping though them. mbuffer, cryptcat, ncat, there are MANY options. Also in term of "safety" (or encryption) many kind of performance depend on how you encrypt your data, and how you compress your transfer because they are pretty…

Certainly. The lftp mirror subsystem allows you to break up batches of files, or even a single file using p-get into multiple streams. Rsync can't do that. Rsync also doesn't work within a chroot environment, whereas lftp does. The exception is if you use native rsync, which means you are not encrypting the stream. You then get chroot, but not privacy. CPU overhead isn't really a limiting factor these days, assuming…

Hum,

> The lftp mirror subsystem allows you to break up batches of files, or even a single file using p-get into multiple streams.

I do not see any particular benefit from that... Network reliability is still a bit needed or you still have to restart...

On restricting rsync on the other end there are many guide so again I do not see much benefit and that's not true that you can't encrypt the transfer otherwise with classic ssh transport this is an example

https://sixohthree.com/1458/locking-down-rsync-using-ssh

but you can do many more things, for instance with cryptcat or ncat in the middle to transport... And they normally run faster than ssh tunnels...

Post reply on HN