Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

neil.computer

Post reply on HN
Print view View original

251–260 of 265 posts

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#251

Post by snapplebobapple »

wolrah wrote: post #168
realityfactchex wrote: post #30

Earlier quoted context omitted.

> Not a great time price-wise to be building a NAS That under-states the matter. It is a terrible time, price-wise, to build a NAS. I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times. (If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging signifi…

> I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times. Almost? ALMOST!?!?! If you handed me a button that would make it like LLMs had never existed I'd be slamming that button so hard Sam Altman's clothes would spin around. Return memory and storage prices to normal, undo the sloppification of ~everything, remove all these annoying "features" that are so useful they have to force t…

I don't really agree with the specifics but I can identify with the emotion. To me the problem with AI isn't AI, it's that investors are idiots and put hundreds of billions of dollars into AI companies that have near zero chance of being profitable enough to pay them back and those companies took that money and blew up computer parts supply chains that are vital to the rest of human existence. If we didn't have this massive build out that has no chance of repaying itself and instead had a build out that only doubled ram/gpu/nvme prices we would be far better off. As it stands, we're going to have a ton of gear that will be of questionable utility once any of the disruptors get their 100X+ per watt improved/different hardware going and to get that hardware we broke everything else. This ai stuff will also likely plateau and have the openweights models and opensource tooling catch up and at that point the companies are basically GPU REITS with diminishing demand as the stuff ages out and the disruptor's gear gets built for much better performance per watt. They've made it really ugly for us now and it will get really ugly for them in the midterm future imo. Not a good outcome for anyone.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#252

Post by dpedu »

I've been re-building my NAS and decided on NFS on top of ZFS. However, my goals are slightly different than the author: I'm using a low-power cpu/motherboard, and I've built 2 distinct systems that can fail over as a form of high availability. I've actually written some custom software to handle the failover and data replication. In my testing, clients see writes block for just a couple second during failover. Would anyone be interested in a writeup about this?

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#253

Post by solarengineer »

regexorcist wrote: post #245
solarengineer wrote: post #223

Earlier quoted context omitted.

When we allocate space to an LVM snapshot, the COW data is written to that space and then when that space is used up the LVM snapshot must either be given more space or must be deleted. When a ZFS snapshot is marked, then changes to existing blocks in the snapshot as well as new writes all go to free space in the disk. There is no pre-allocated space within which these COW changes are written to. Thus, we could have…

That would be a thinly-provisioned snapshot in LVM.

Thank you. I went back to the link I shared and saw that there is section just below on thin provisioning via snapshots.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#254

Post by Our_Benefactors »

randusername wrote: post #153
realityfactchex wrote: post #30

Earlier quoted context omitted.

> Not a great time price-wise to be building a NAS That under-states the matter. It is a terrible time, price-wise, to build a NAS. I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times. (If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging signifi…

I'm doing it anyway and I've found some secondhand deals. Maybe COVID homelabbers are offloading gear? I built a 24 TB HDD NAS in a 36 bay chassis for around $1000 all-in: mb, ram, rails, chassis, rack, disks, hba, nic Awful Watts/TB but the plan is to run it and the GPU rig on solar.

What size discs are you using? This post is kind of hilarious, 24TB is achievable in literally a 2-bay NAS nowadays.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#255

Post by bensyverson »

Dylan16807 wrote: post #228
bensyverson wrote: post #201

Earlier quoted context omitted.

The risk of z1 is that if you get a read error during resilvering, that data is permanently corrupted. The odds of this happening go way up the larger your individual drives are. This is why I chose RAIDZ2 for my NAS. I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.

> The odds of this happening go way up the larger your individual drives are. This is a common claim, but honestly, citation needed. We can't just apply bit error rates from the datasheet that haven't been updated in 15 years. I'm sure a 2-day rebuild is a little more risky than a 4-hour rebuild, but I'm not convinced it's by all that much. Especially if you had a monthly scrub going to prevent disk rot and disks sec…

> This is a common claim, but honestly, citation needed.

This is based on the data I had available 10 years ago when I set up this array… If you have data that points to drives having fewer read errors per GB in recent years, I'd love to see that citation.

> And how many times did you have a read error on one of your other disks during a resilver?

I don't know, because I have RAIDZ2, so that read error would probably be corrected without it being reported to me.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#256

Post by bensyverson »

bell-cot wrote: post #232
bensyverson wrote: post #201

Earlier quoted context omitted.

The risk of z1 is that if you get a read error during resilvering, that data is permanently corrupted. The odds of this happening go way up the larger your individual drives are. This is why I chose RAIDZ2 for my NAS. I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.

> The odds of this happening go way up the larger your individual drives are. Technically true. Practically? There are several other major factors - such as the quality of your drives, whether you periodically run zfs scrubs and/or disk read tests, and how long a resilver takes to complete. Behind those are the eternal duo - size of budget and cost of failure.

[dead]

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#257

Post by Dylan16807 »

bensyverson wrote: post #255
Dylan16807 wrote: post #228

Earlier quoted context omitted.

> The odds of this happening go way up the larger your individual drives are. This is a common claim, but honestly, citation needed. We can't just apply bit error rates from the datasheet that haven't been updated in 15 years. I'm sure a 2-day rebuild is a little more risky than a 4-hour rebuild, but I'm not convinced it's by all that much. Especially if you had a monthly scrub going to prevent disk rot and disks sec…

> This is a common claim, but honestly, citation needed. This is based on the data I had available 10 years ago when I set up this array… If you have data that points to drives having fewer read errors per GB in recent years, I'd love to see that citation. > And how many times did you have a read error on one of your other disks during a resilver? I don't know, because I have RAIDZ2, so that read error would probably…

> This is based on the data I had available 10 years ago when I set up this array

Data you collected or data you found? Any idea where it was?

I've only seen oversimplified math, not real data, and different versions of the math make different assumptions.

> I don't know, because I have RAIDZ2, so that read error would probably be corrected without it being reported to me.

Zpool status gives you a per-disk error count. I kind of assumed someone doing a resilver would use that command.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#258

Post by bensyverson »

Dylan16807 wrote: post #257
bensyverson wrote: post #255

Earlier quoted context omitted.

> This is a common claim, but honestly, citation needed. This is based on the data I had available 10 years ago when I set up this array… If you have data that points to drives having fewer read errors per GB in recent years, I'd love to see that citation. > And how many times did you have a read error on one of your other disks during a resilver? I don't know, because I have RAIDZ2, so that read error would probably…

> This is based on the data I had available 10 years ago when I set up this array Data you collected or data you found? Any idea where it was? I've only seen oversimplified math, not real data, and different versions of the math make different assumptions. > I don't know, because I have RAIDZ2, so that read error would probably be corrected without it being reported to me. Zpool status gives you a per-disk error coun…

> Data you collected or data you found? Any idea where it was?

I think it was a lengthy blog post citing manufacturer data, found via the FreeNAS forums. The added safety of RAIDZ2 was a good tradeoff. Someone else on this thread reports they lost two SSDs in rapid succession, which I hear happens sometimes with drives from the same batch. In any event, I felt no need to launch an independent investigation of the math and underlying assumptions.

> Zpool status gives you a per-disk error count. I kind of assumed someone doing a resilver would use that command.

Yes, I did indeed use zpool to do the resilver… I don't recall if there were any read errors on other drives at the time of those failures, but probably not, as the scrub probably would have flagged it. They've all been replaced with higher capacity drives, so at the moment I see "No known data errors"

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#259

Post by Dylan16807 »

bensyverson wrote: post #258
Dylan16807 wrote: post #257

Earlier quoted context omitted.

> This is based on the data I had available 10 years ago when I set up this array Data you collected or data you found? Any idea where it was? I've only seen oversimplified math, not real data, and different versions of the math make different assumptions. > I don't know, because I have RAIDZ2, so that read error would probably be corrected without it being reported to me. Zpool status gives you a per-disk error coun…

> Data you collected or data you found? Any idea where it was? I think it was a lengthy blog post citing manufacturer data, found via the FreeNAS forums. The added safety of RAIDZ2 was a good tradeoff. Someone else on this thread reports they lost two SSDs in rapid succession, which I hear happens sometimes with drives from the same batch. In any event, I felt no need to launch an independent investigation of the mat…

Okay, interesting. Also to be clear I do think Z2 makes sense in a lot of situations, but I suspect that it's not very dependant on drive size like a lot of people think. If I wanted high reliability I'd use Z2 even with small drives.

Also to reply to your other comment: Personally one extra drive is a small price to pay for the peace of mind.

I achieve peace of mind via backups. For each copy, once I get the AFR below 2% I'm not too fussed about pushing it lower.

Re: How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

#260

Post by turpentine »

ahepp wrote: post #117
anon-3988 wrote: post #77

I really, do not want to have a big server rack lying around at my house. I have a couple of SSDs that are not being used and 3 USB HDD drives. I also have a PN50 that I am not using as well. I am thinking of buying a USB bay with 5 SSD slots in it and then 3 HDD drive. My use case is very, very cold. It is mostly just readonly data with some rclone sync every week. Does anyone have a suggestion? I am pretty much rel…

I'd probably just buy a cheap prebuilt PC with a big case, or surplus enterprise tower. You can get cheap LSI PCI cards and do miniSAS to SATA if needed. The biggest concern I'd have with USB is power delivery to the hard drives, but I haven't even done the napkin math so maybe it's fine. The SSDs seem like they might be a waste of money. USB hard drives have a poor reputation, but I don't have a ton of experience to…

> You can get cheap LSI PCI cards and do miniSAS to SATA if needed.

People say those generate a ton of heat because they're normally in a rack mounted server with plenty of airflow. Probably a significant drawback for a "cheap prebuilt PC".

Post reply on HN