Live data from Hacker News

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

neil.computer

141–150 of 265 posts

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#141
post #104
post #95

Earlier quoted context omitted.

First: Do you have backups of your single high-capacity HDD? That's my biggest worry. What's your plan for when that HDD fails one day, as it will? The main reasons people go for ZFS and a "NAS" is checksums and data integrity protection, as well as maybe not wanting to keep their primary machine awake all the time (I personally don't). Then there are useful features like snapshots, which means I don't have to worry…

I have external parity for everything with rsync. I don't have any storage that isn't matched with external backups. Fortunately I've never seen any bitrot or had a drive fail on me since I tend to replace them before they do. And yeah, I could add in some extra HDDs for RAID, but this is more a personal choice because of how much I hate SATA. Filesystems like ext4 are because I value boring and rock solid stability…

How do you know you haven't seen bitrot? What rsync arguments do you use to error out when an old, untouched file suddenly changes?

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#142
post #57
post #38

Earlier quoted context omitted.

ZFS makes this completely trivial except for the "beeps and lights a led" part

smartmontools makes this trivial and can email you which might be better than beeping and lights if you’re not always near your server. I use that plus smartctl_exporter so I can do metrics and alerting off that.

smartmontools misses data corruption at the drive layer, as it trusts the drives to report faithfully.

"In hashes I trust"

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#143

Earlier quoted context omitted.

I would kill (figuratively, anyway) to have no AI whatsoever. No slop machine threatening to replace my job, or turn my job into babysitting its stupidity, and hardware would be reasonably priced? That would be awesome. AI has brought me nothing but downside.

AI has freed most people I know from the tedious job of writing actual emails and birthday greetings - leaving then more time to e.g. wash dishes or clean the floor. Seriously, if arts and creativity is what sets humans apart from other animals, then AI has almost completely displaced our capacity to even consider doing these activities ourselves. People reach for AI when they should be composing a birthday greeting…

People were Googling these things. Those who were good with words, didn’t do that before AI and won’t do it today. They don’t have to.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#144
post #117

I really, do not want to have a big server rack lying around at my house. I have a couple of SSDs that are not being used and 3 USB HDD drives. I also have a PN50 that I am not using as well. I am thinking of buying a USB bay with 5 SSD slots in it and then 3 HDD drive. My use case is very, very cold. It is mostly just readonly data with some rclone sync every week. Does anyone have a suggestion? I am pretty much rel…

I'd probably just buy a cheap prebuilt PC with a big case, or surplus enterprise tower. You can get cheap LSI PCI cards and do miniSAS to SATA if needed. The biggest concern I'd have with USB is power delivery to the hard drives, but I haven't even done the napkin math so maybe it's fine. The SSDs seem like they might be a waste of money. USB hard drives have a poor reputation, but I don't have a ton of experience to…

The big USB racks all have separate power.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#145
post #76

Earlier quoted context omitted.

AI has freed most people I know from the tedious job of writing actual emails and birthday greetings - leaving then more time to e.g. wash dishes or clean the floor. Seriously, if arts and creativity is what sets humans apart from other animals, then AI has almost completely displaced our capacity to even consider doing these activities ourselves. People reach for AI when they should be composing a birthday greeting…

Is it a genuine birthday greeting if you just let AI do the job?

that was their point

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#146
post #64

Earlier quoted context omitted.

I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real? Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the…

> I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real? I'm not entirely sure, but it seems to me that free space (and the 20% reservation) is mostly a proxy for fragmentation, and you can therefore better look at fragmentation directly. That would mean that if you mostly store large files, there shouldn't be a lot of fragmentation even at hig…

> It's also something that probably isn't too different from the performance of other filesystems at high utilization, so it shouldn't be exaggerated.

Well, other filesystems can defragment.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#147

Earlier quoted context omitted.

https://diskprices.com/ is great for this You have SATA or SAS to pick from. The CPU requirements for a storage server are not high. On a typical ATX board you have motherboard SATA and can put SAS controllers in the spare PCIe slots. My first "NAS" was two 22TB hard drives in a ZFS pool on my motherboard SATA

ATA (SATA), SCSI (SAS), and NVM (NVMe).

no. just sata & sas. There is no nvme spinning drive and flash loses bits just from sitting, it's not archival. flash is good for working not for storing.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#148
post #111

Earlier quoted context omitted.

I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real? Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the…

For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.

This isn’t black and white. You might have huge amounts of non-essential data that aren’t worth the cost of off-site backups, but worth the cost of an extra disk of redundancy to lessen the risk. Even when you do have backups, it will reduce the risk of extended downtime (and possibly egress costs) caused by having to restore large amounts of data from backups.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#149
This is pretty much what I did, except that since I use Proxmox I used that to manage the volumes and then just did a pass-through mount to an LXC that runs Samba - that way I get a decent, simple GUI to monitor and manage volumes and can do user management inside the LXC.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#150
post #104
post #95

Earlier quoted context omitted.

First: Do you have backups of your single high-capacity HDD? That's my biggest worry. What's your plan for when that HDD fails one day, as it will? The main reasons people go for ZFS and a "NAS" is checksums and data integrity protection, as well as maybe not wanting to keep their primary machine awake all the time (I personally don't). Then there are useful features like snapshots, which means I don't have to worry…

I have external parity for everything with rsync. I don't have any storage that isn't matched with external backups. Fortunately I've never seen any bitrot or had a drive fail on me since I tend to replace them before they do. And yeah, I could add in some extra HDDs for RAID, but this is more a personal choice because of how much I hate SATA. Filesystems like ext4 are because I value boring and rock solid stability…

What do you use for external parity? Par2?

If you were to replace your current drive with a new larger one and your RAM or your SATA controller silently corrupted the files how would things play out? Would these corrupted files propegate to the backups?

Ext4 is battle tested so I understand your reasoning. I think you just need to figure out a way to detect silent corruption and a way to snapshot your files in case they do get corrupted.

From experience I can tell you that it is an absolute pain having to manually sort through a bunch of files trying to detect which ones are corrupt and which ones are good.

Post reply on HN