How I store my files • Hacker News Board

Re: How I store my files

#61

Post by poisonborz » Fri, Jul 30, 2021, 4:03 PM UTC

mnemnc wrote: post #52

I feel like I have a fairly solid backup system for all of my important files. They are protected against elementary disasters and cyber attacks alike. However the only thing I struggle with are my phone photos. I recently caved in and started using iCloud Photos due to the convenience of having all my phone photos alwas available, searchable and tagged, even if the library size exceeds my phone‘s capacity. Does anyo…

I do this with Syncthing (not just photos, but all phone files), a very customizeable file sync tool, it can do trash bin and versioning similar to cloud storage providers. As I heard sadly it doesn't work so well in iOS due to its restrictions though.

Re: How I store my files

#62

Post by mceachen » Fri, Jul 30, 2021, 4:04 PM UTC

gen3 wrote: post #47

shreddit wrote: post #45

Earlier quoted context omitted.

Doesn’t thumbdrive data decay over time? Like all flash based storage types do

That is my understanding. I've heard numbers around 10yr +-5, but I've never seen a study on it. I wonder if storing them zip-locked in a freezer would help (but I guess you could just use a magnetic hardrive :D )

That the number I've personally experienced (I also replied to GP).

It's an interesting idea to deep-freeze your drives, but the thermal stress from modern frost-free freezers that automatically dethaw periodically may negate the benefit of chilly bits...

If you don't have that much to store, and are worried about data durability, I'd suggest looking into M-disc instead.

Re: How I store my files

#63

Post by IggleSniggle » Fri, Jul 30, 2021, 4:05 PM UTC

mnemnc wrote: post #52

I feel like I have a fairly solid backup system for all of my important files. They are protected against elementary disasters and cyber attacks alike. However the only thing I struggle with are my phone photos. I recently caved in and started using iCloud Photos due to the convenience of having all my phone photos alwas available, searchable and tagged, even if the library size exceeds my phone‘s capacity. Does anyo…

I spent extensive time evaluating this problem, and tried a whole bunch of different things:

Nextcloud, Resilio, iCloud, etc etc etc.

Honestly? Just using Google Photos is probably the best bet for most folks, whether you’re on iOS or something else. I personally picked OneDrive for my use case since it is both more performant for large data sync and 6 TB + Office 365 for a family plan beat out the alternatives on pricing.

Re: How I store my files

#64

Post by mceachen » Fri, Jul 30, 2021, 4:07 PM UTC

noveltyaccount wrote: post #51

ga3h wrote: post #44

Thumb drives. Financial stuff, copies of legal documents, large libraries of PDF books, reports, etc., etc. And then back the main drive up to FOUR thumb drive backups. Reasons: (1) USB drives are huge now. (2) Convenient - can move from one PC to another easily. (3) Convenient - I do NOT have time to mess with other solutions. (4) Safety - valuable financial stuff almost always OFFLINE reducing hacker vulnerability.…

I'll add - drives from different brands. I had a mirrored RAID 1 fail at one point (in the '00s, spinning magnetic hard drives), two identical drives, and they failed within days of each other. One drive died, I ordered a replacement, and before the replacement arrived the second drive died. The shock and disbelief I felt of my foolproof RAID solution going belly-up on me! And I still can't believe I didn't turn off…

This is good advice, and isn't just anecdata: see "correlated failures": https://photostructure.com/faq/raid-is-not-a-backup/#4-data-...

Re: How I store my files

#65

Post by mceachen » Fri, Jul 30, 2021, 4:11 PM UTC

IggleSniggle wrote: post #63

mnemnc wrote: post #52

I feel like I have a fairly solid backup system for all of my important files. They are protected against elementary disasters and cyber attacks alike. However the only thing I struggle with are my phone photos. I recently caved in and started using iCloud Photos due to the convenience of having all my phone photos alwas available, searchable and tagged, even if the library size exceeds my phone‘s capacity. Does anyo…

I spent extensive time evaluating this problem, and tried a whole bunch of different things: Nextcloud, Resilio, iCloud, etc etc etc. Honestly? Just using Google Photos is probably the best bet for most folks, whether you’re on iOS or something else. I personally picked OneDrive for my use case since it is both more performant for large data sync and 6 TB + Office 365 for a family plan beat out the alternatives on pr…

Eh... I'd definitely do a Google Takeout to verify everything's OK before you delete your originals.

You may think the downsampled image is sufficient for lossy backups, but older images stored in "high quality" in my personal GP library were almost wholly stripped of metadata, including when the photo was taken.

(Many of my users have suffered from this as well, which is why I built tag value inference into PhotoStructure to try to help spackle over these metadata holes).

Re: How I store my files

#66

Post by bachmeier » Fri, Jul 30, 2021, 4:14 PM UTC

> Everything can look really nice "on paper" but you don't know what goes on behind the scenes. I have worked with a lot of different people and I have seen too much crazy shit to fully trust anyone with my important data. A cloud provider may have the best of intentions, but sometimes all it takes is a single grumpy employee or even a minor mistake to do a lot of damage.

OneDrive and Google Drive are both pretty cheap. Is there anything wrong with keeping a backup of your important data in one of them? At a certain point you have to live your life and take a chance. Sure, I never made it to Italy, but I had a 100% safe backup system for my files said noone ever.

> Free Git hosting such as GitHub, GitLab and others can also be utilized for data that you don't mind storing in public. GitLab and other providers does provide free private repositories, just don't rely fully on that.

At this point, it's clear the author is looking for arguments to make. Of course you're not going to dump all your stuff into a GitLab repo in the cloud. You're going to clone it on multiple machines! My important work stuff is under version control and cloned on multiple machines in multiple locations. If that's not good enough, I'll live with the consequences.

Re: How I store my files

#67

Post by IggleSniggle » Fri, Jul 30, 2021, 4:19 PM UTC

mongol wrote: post #13

Something tricky is how to bootstrap restoration of backup. If you have lost "everything", how do you get it back? For example, if you use borg to backup remotely via ssh, you will need ssh keys as well as a passphrase for the encrypted backup. Where do you store those to make sure you have them if your computer is gone? What I did was create a self-extracting restoration script, which embedded everything needed. Thi…

I keep the most important pieces on printout in a firesafe lockbox. I don’t have any eventuality for if the firesafe goes at the same time as online failsafes, but I feel like the best approach for those kinds of situations is nihilism.

Re: How I store my files

#68

Post by 90minuteAPI » Fri, Jul 30, 2021, 4:24 PM UTC

dangerface wrote: post #60

> These tools work really great, until they don't! Huh? Whats the problem? How is ZFS any different because its lower level? > ZFS without ECC memory is no worse than any other file system without ECC memory. You really do need ECC memory unless you are ok with your pool becoming corrupt every 6 months or so. Im just saying this from my experience of running a ZFS server without ECC, the data wasn't critical so I lef…

I was so tempted to jump on this with my anecdata of successfully running ZFS without ECC for a time, but that was in the context of a dirt cheap home tinker box.

If you care about data integrity enough to use ZFS, just bite the bullet and use ECC. If you care about data integrity at all, use ECC. If you don’t, what are we even doing here?

Re: How I store my files

#69

Post by ajford » Fri, Jul 30, 2021, 4:25 PM UTC

Gurdt wrote: post #12

Can someone point me to the flaws of my method? I simply use Google Drive. Me and my partner have multiple devices (tablet, iPhone, Android phone, multiple Mac and Windows laptops) and we just sync folders to our desktop. We just store everything we don't want to lose in Drive. We share folders we both need. Photo's we take on our phones are automagically backed up in the cloud, music and movies are streamed. My hous…

Makes you solely dependent on Google. That's not really a problem, but as we've seen many times on HN and elsewhere, there's always the random chance that Google's automated systems can decide that your account meets some unknown criterion to be locked, deleted, or otherwise preventing you access.

It's rather unlikely, but so is the chance of your house burning down.

That's generally the reason people recommend the 3-2-1 rule. By having multiple separate backup solutions, you're hedging your bets. But as is commonly the case in computing, you have to play the convenience vs security/reliability game here and decide where the line between your time and convenience lies against the reliability of your backups and what risks your willing to take.

Re: How I store my files

#70

Post by IggleSniggle » Fri, Jul 30, 2021, 4:29 PM UTC

mceachen wrote: post #65

IggleSniggle wrote: post #63

Earlier quoted context omitted.

I spent extensive time evaluating this problem, and tried a whole bunch of different things: Nextcloud, Resilio, iCloud, etc etc etc. Honestly? Just using Google Photos is probably the best bet for most folks, whether you’re on iOS or something else. I personally picked OneDrive for my use case since it is both more performant for large data sync and 6 TB + Office 365 for a family plan beat out the alternatives on pr…

Eh... I'd definitely do a Google Takeout to verify everything's OK before you delete your originals. You may think the downsampled image is sufficient for lossy backups, but older images stored in "high quality" in my personal GP library were almost wholly stripped of metadata, including when the photo was taken. (Many of my users have suffered from this as well, which is why I built tag value inference into PhotoStr…

That’s good advice for those that care, and I should have mentioned it, especially for HN crowd. I definitely noticed this problem when I did Google Takeout on my photos dating back to the Picasa days.

I still think that for most people (and on HN, where we care more about fighting data entropy, there might be significantly lower overlap with the majority), Google Photos is the best option.