Live data from Hacker News

How I store my files

unixsheikh.com

121–130 of 144 posts

Re: How I store my files

#121

What's the best way to encrypt files, GunPG, openssl, ccrypt, some AES256 command line utility, or some backup can do it automatically? I do want to encrypt part of my backup before upload them to the cloud.

I don't know what's the best, but rclone looks like the simplest option https://rclone.org/crypt/

Re: How I store my files

#122
post #96

I'm a digital nomad, and I decided I wanted to store all of my data as securely as possible without relying on Google/Amazon/Microsoft/Apple. Here's my setup, in case anyone finds it useful. I store all of my data in Nextcloud hosted on a VPS. It's virtually impossible to guarantee the security of data on a running VPS, so my sensitive data is also encrypted with Cryptomator, so I don't have to trust my VPS host (but…

Wondering something, if you are using cryptomator and a VPS - so you don't own your data and use a higher level encryption - why not using the cloud storage services (i.e Dropbox, icloud etc.) ?

There are two reasons I don't use Dropbox, iCloud, etc.

First, I want to be able to use open source, audited clients that allow me to access my data in a transparent and flexible way on all of my devices. I don't want to use a closed source sync client that doesn't give me full control over how my data is synced.

Second, and more important, if you don't control your encryption keys, you don't control your data.

Storing my data on a VPS doesn't mean that I don't own it. I still have full control over it, and if the VPS disappears, I still have redundant copies.

Re: How I store my files

#123

I also have a ZFS NAS and a separate backup server for the NAS that only gets turned on once a week. I disagree about his recommendation to avoid encryption. I keep the NAS and backup server drives encrypted. It’s not a big deal to unlock the drives at boot with a password and I’ve never had any issues with it. If someone breaks in and steals one of the servers, at least they won’t have access to years of all of my d…

I have something simpler than yours, with a small Proxmox server as my NAS (no ZFS) and backups stored to a second disk. Occassionally I will backup the contents of that machine to external drives.

In the future I plan on doing something similar to what you have, with a powerful ZFS server and my current server accessing the main server using a read-only interface as a second backup. It does take work to maintain, and I don't look forward to the day when I have a full server failure and need to scramble to replace the unit, restore from backups, and all that. Ideally I would have a pair of servers to handle that eventuality... and down the rabbit hole we go.

Ultimately, I guess this all is a tradeoff - I know many engineers who swear on their cloud services, and they simply don't have the time and experience to maintain dedicated servers. While Synology devices and FreeNAS make things simpler, that's for a best-case scenario as you need the technical knowledge to deal with the issues that will eventually pop up. Honestly the cloud is what I recommend for all but the most technical folk, with the addition of external disks off-site for the most critical files.

Re: How I store my files

#124

> These tools work really great, until they don't! Huh? Whats the problem? How is ZFS any different because its lower level? > ZFS without ECC memory is no worse than any other file system without ECC memory. You really do need ECC memory unless you are ok with your pool becoming corrupt every 6 months or so. Im just saying this from my experience of running a ZFS server without ECC, the data wasn't critical so I lef…

ZFS doesn't need ECC but it does benefit from it. If your pool was becoming corrupt every six months, look to a shitty drive controller or cabling first.

I've run ZFS for quite a few years now on laptops and lower-end machines that can't use ECC memory and I've never had corruption, unrecoverable files/pools, etc.

https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-y...

Re: How I store my files

#125
post #41

>Not only does encryption during data recovery make everything much more difficult, but should you pass away, your family members might not have the skills required to access the data Uh, trust me, that's a good thing.

If I can't delete my browsing history before I pass on, you bet your bottom dollar I'm not letting my family have my keys!

Or don't keep a browsing history, I just keep better bookmarks.

Re: How I store my files

#126
post #85

Earlier quoted context omitted.

How common an occurence is that? How often is an unimportant, middle class person's data at risk, really? Enough that you'd want to spin up your ZFS storage? Hypothetically, let's say I had my entire life on Google. I have a unique password for it, backed up by 2FA, without the SMS/Authenticator fallback. What's the long term consequence? Google knows everything about me? They already do anyway. Someone can steal my…

Everyone has their own opinions on this and their own threat vectors for their own personal situation. The following is my opinion based on my own situation which I believe to apply to the average person: I think it is safe to assume that Google does some sort of data mining on the data you upload. If that bothers you, self hosting everything isn't your only option -- you can also encrypt everything before uploading…

To add to that, similar thing happened with the Terraria dev [1]. if things like this can happen to him, I don't know where I stand.

[1] https://twitter.com/demilogic/status/1358661840402845696?lan...

Re: How I store my files

#127
I really like this guy's web page source. Do a source view, it's all very neat and tidy, you could just read that and not feel you're losing much in terms of experience.

Re: How I store my files

#128

> These tools work really great, until they don't! Huh? Whats the problem? How is ZFS any different because its lower level? > ZFS without ECC memory is no worse than any other file system without ECC memory. You really do need ECC memory unless you are ok with your pool becoming corrupt every 6 months or so. Im just saying this from my experience of running a ZFS server without ECC, the data wasn't critical so I lef…

I stored 2tb of actively used data on zfs without ecc ram from freebsd 8 to 12. I had no noticeable corruption. I only post this because there is a weird assumption that not using ecc ram is a death sentence for your zfs data. I use ecc ram now and I think it's the proper way to do zfs, but let's not pretend you forfeit your data by not using ecc.

ECC RAM is useful for ALL FILESYSTEMS - not just only for ZFS.

But from all filesystems ZFS works best of all when you do not have ECC RAM.

The 'requirement' that ZFS needs ECC RAM to work properly is one of the biggest myths of the Internet ...

Re: How I store my files

#129
post #117

Earlier quoted context omitted.

Got it. So perhaps after the Duplicity files have been incrementally uploaded to the remote datacenter, for the cold storage backup of those files rather than simply duplicating the Duplicity files I should unpack them and then rearchive them into a single flat encrypted archive.

>>I should unpack them and then rearchive them into a single flat encrypted archive. Which is a duplicity full back up (that one would normally do once a month or so). Deduplication is an additional potential point of failure.

I am hoping to do a Duplicity Full Backup rarely due to data caps. I am hoping I can unpack the incremental Duplicity archives and only if it fails verification will I do a Full Backup.

Re: How I store my files

#130
> While you might consider doing a full encryption for both your personal laptop and/or desktop, in case one of these gets stolen, you should avoid encryption on backup and storage when it really isn't needed because encryption adds yet another layer of complexity.

Wrong. Not encrypting your backups when your daily use systems are encrypted makes no sense. Seriously. Why go through the hassle of figuring out a password, when one can steal the unencrypted off site backup?

You want simple encryption? Me too! Use dm-crypt/LUKS. It's been in every modern Linux distro for at least 10 years. If you can plug your encrypted external drive into a live/freshly installed Linux desktop and a encryption prompt comes up, you're in luck!

Post reply on HN