Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

neil.computer

Post reply on HN
Print view View original

221–230 of 265 posts

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#221

Post by throwaway270925 »

bobmcnamara wrote: post #140
traches wrote: post #129

Earlier quoted context omitted.

ZFS handles drive failures more robustly than anything else. There's a reason synology uses mdadm under BTRFS instead of the built-in BTRFS RAID features, and mdadm operates at the device level. That means that to replace a drive, mdadm has to rebuild the entire drive while zfs will only rebuild what's actually in use. > Also, any guide like this that doesn’t guide you through “disk 3 failed, this is how you safely r…

The worst part of either approach is: how do I find disk number 39 out of 48.

By using /dev/disk/by-id when adding them to the pool - and looking at the physical label on the disk itself. All disks I ever worked with had that ID printed somewhere on there.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#222

Post by throwaway270925 »

mianos wrote: post #131
bradley13 wrote: post #22

Just yesterday I was brainstorming with ChatGPT about this. I have an ancient QNAP plus a slightly less ancient NUC running PiHole, Wireguard and other services. Both need replaced, so why not combine them? I don't know much about ZFS, but it sounds like I need to learn. Docker may have conquered the world, but I plan to stay with LXD for services. The one thing I take issue with: an appliance like this runs 24/7. It…

I run zfs as the storage pool for my incus (next lxd) services. It is the ideal fit. Here is a list: - Instant, zero-copy container cloning from images via Copy-on-Write. If you boot a new image like the existing ones it's seconds. - Atomic, millisecond-level instance snapshots regardless of storage size - Block-level container migration using native 'zfs send' and 'zfs receive', very short command lines and seems to…

> - Dedup keeps blocks with the same data as a reference. This costs a lot of memory

To get around this you can add an SSD mirror as "Special" vdev, the dedup table will then be saved there, freeing up your memory.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#223

Post by solarengineer »

regexorcist wrote: post #207
solarengineer wrote: post #197

Earlier quoted context omitted.

"It is important to regularly monitor the snapshot’s storage usage. If a snapshot reaches 100% of its allocated space, it will become invalid." [1] ZFS snapshots use Copy-on-Write differently and have no such allocated space limit. Thus, we can do interesting things like snapshotting a file system after an OS is installed, and then roll back to that snapshot upon an OS upgrade failure, or even clone a new file system…

You can do those things with LVM snapshots too, that's what they are used for.

When we allocate space to an LVM snapshot, the COW data is written to that space and then when that space is used up the LVM snapshot must either be given more space or must be deleted.

When a ZFS snapshot is marked, then changes to existing blocks in the snapshot as well as new writes all go to free space in the disk. There is no pre-allocated space within which these COW changes are written to.

Thus, we could have a single ZFS snapshot marked today followed by years of data being written to. This is not the intent of an LVM snapshot and the warning that I had quoted above is worth reviewing once.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#224

Post by jsr0 »

Confiks wrote: post #7

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA backplane and drive bays (unlike the earlier Jonsbo variants). I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-fil…

Have you looked at https://zfs.rent/ for cloud storage? You have to provide your own drive, but the monthly cost is relatively low.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#225

Post by devy »

Confiks wrote: post #7

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA backplane and drive bays (unlike the earlier Jonsbo variants). I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-fil…

Yep. The 4 Samsung SSD 990 PRO 4TB NVMe SSDs are $1100 apiece. Yikes!

https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-p...

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#226

Post by zymhan »

ocd wrote: post #91

I'm sure people have reasons for taking these things as far as they do with ZFS, and everything else that goes into what is commonly considered a "NAS." But I've found great convenience in a tight NFSv3 config running from a single high capacity HDD with ext4 on a Linux machine (primary system that's always on anyway while I'm around/awake) making things available to my other devices for the electricity and overhead…

Once you add multiple machines reading and writing simultaneously, you'll hit a limit. But that's fine, build what you need now, no need to over-optimize.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#227

Post by tomasff »

Confiks wrote: post #7

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA backplane and drive bays (unlike the earlier Jonsbo variants). I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-fil…

> So now I have a large script which debootstraps a Debian based NAS into being.

Is part of this publicly available? After seeing, - https://words.filippo.io/frood/ - https://0pointer.net/blog/fitting-everything-together.html

I've been looking at setting something up with mkosi/systemd for my home server/NAS - it's interesting seeing how different setups are solving the problem for inspiration.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#228

Post by Dylan16807 »

bensyverson wrote: post #201
cm2187 wrote: post #111

Earlier quoted context omitted.

For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.

The risk of z1 is that if you get a read error during resilvering, that data is permanently corrupted. The odds of this happening go way up the larger your individual drives are. This is why I chose RAIDZ2 for my NAS. I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.

> The odds of this happening go way up the larger your individual drives are.

This is a common claim, but honestly, citation needed.

We can't just apply bit error rates from the datasheet that haven't been updated in 15 years. I'm sure a 2-day rebuild is a little more risky than a 4-hour rebuild, but I'm not convinced it's by all that much. Especially if you had a monthly scrub going to prevent disk rot and disks secretly getting super fragile.

> I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.

And how many times did you have a read error on one of your other disks during a resilver?

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#229

Post by Dylan16807 »

Brian_K_White wrote: post #147
dapperdrake wrote: post #88

Earlier quoted context omitted.

ATA (SATA), SCSI (SAS), and NVM (NVMe).

no. just sata & sas. There is no nvme spinning drive and flash loses bits just from sitting, it's not archival. flash is good for working not for storing.

A NAS is plugged in. Flash works fine for it.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#230

Post by Dylan16807 »

toast0 wrote: post #205
dijit wrote: post #74

Earlier quoted context omitted.

Usually if you’re using a NAS you don’t want to lose data, ZFS is not significantly more sensitive than everything else. But everything is actually quite sensitive. We’ve accepted lack of ECC because Intel decided it would be a product line differentiator, and serious customers who didn’t want random crashes or to lose data would buy chips with ECC. It’s actually less of an issue these days because DDR5 has (by spec)…

> We’ve accepted lack of ECC because Intel decided it would be a product line differentiator, and serious customers who didn’t want random crashes or to lose data would buy chips with ECC. AMD has been allowing ECC on lots of regular hardware for a long time. People don't tend to buy ECC for desktop use because it costs significantly more (used server ram is/was often cheap... but it often doesn't work in desktop boa…

> AMD has been allowing ECC on lots of regular hardware for a long time.

Often unsupported or untested by the motherboard manufacturer, because the precedent set quite a while ago has stood so strongly.

> because it costs significantly more [...] and the performance specs are poor

Because it's niche. If a large share of the desktop market was using ECC memory, the extra cost would have been 10-12%, and a typical kit would push the clocks and timings at least as much as non-ECC memory.

> Since DDR5 doesn't include reporting on bit errors (afaik), it likely means much fewer single bit errors, but most experienced errors will be multi-bit. Although, I dunno what proportion of bit errors is on the ram chips and what's on the bus... there's no protection from bus errors.

Yeah that sucks. I'm really hoping that DDR6 standardizes a light form of end to end ECC. (Bursts are 24 bits wide and 24 bits deep, so on top of the 512 bits of payload there's a bunch of extra bits that can be used for ECC. If memory chips would just store 4 extra bits per 128, that's enough for reasonable ECC on a per-burst basis.)

Post reply on HN