Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Budget Homelab NAS Server

mtlynch.io

Post reply on HN
Print view View original

231–240 of 316 posts

Re: Building a Budget Homelab NAS Server

#231

Post by zaarn »

vetinari wrote: post #129
zaarn wrote: post #47

Earlier quoted context omitted.

I don't think BTRFS supports NFSv4 ACLs yet (ie, Windows ACLs are natively supported on ZFS, there is patchset so Linux also supports it but BTRFS obviously has no integration for a patchset that only exists for ZFS). Having NFSv4 ACL access is a huge plus since you can configure permissions natively from windows and have them enforced even on the shell.

Not sure how Synology implemented it, but they do support Windows ACLs on btrfs volumes.

They likely use XATTRs to store the ACL (that is an option in Samba), but it's not native like it's on the TrueNAS systems with the kernel. I bet if you log into the Syno's via SSH you don't get the ACLs enforced on the shell. With the NFSv4 ACL patchseries, they would and you could benefit from the better options that the NFSv4 ACLs give you.

Storing them in metadata is not the same as having them natively.

Re: Building a Budget Homelab NAS Server

#232

Post by mobilio »

RobLach wrote: post #229

If you're going for budget, decommissioned data center rack servers work well. I purchased a dual xeon (24 cores total) with 64gb of memory, 12 3.5" bays, dual power supplies, for about $250 from a liquidator. Filling it with HDDs was pricey but you can expand as you need it to spread the expenditure.

Same here - 32 cores/64G ram/4 3.5" bays, dual power 460w for $300.

Results: - idle - 55W - full usage - 200W

Not bad for 10 years old server.

Re: Building a Budget Homelab NAS Server

#233

Post by walterbell »

Are there current JBOD products in 1U short-depth (15" for wall mounted rack) form factor, e.g. 4 x 3.5" hotswap drive bays with a mini-SAS connection to the unit? This would be useful as a backup device, or low-power NAS when connected to a Linux thin-client with LSI HBA.

There were some 1U products which included RAID support, priced around $500, which is a bit much for 1U chassis + SATA/SAS backplane + Pico power supply. 1U chassis with ~11" depth (seems to be a telco standard?) start around $100.

StarTech 1U JBOD discontinued, https://www.startech.com/en-us/hdd/sat35401u

Silverstone RS431 JBOD unavailable, https://www.silverstonetek.com/product.php?pid=482&area=en

iStarUSA upcoming (shipping in June) 1U JBOD is $400, http://www.scsi4me.com/istarusa-m-140ss-jb-1u-3-5-4-bay-tray...

For ~$600, QNAP has an Arm-based 1U short-depth NAS with 2x10GbE and 2x2.5GbE networking, plus dual M.2 NVME slots. Maybe Armbian will run on that SoC, it's part of a supported family. https://www.qnap.com/en-us/product/ts-435xeu

$100 ODROID M1 SBC has an M.2 NVME slot with 4x PCIe lanes. In theory, this could be bridged to a PCI slot + LSI HBA, within a small case, as a DIY low-power NAS.

Re: Building a Budget Homelab NAS Server

#234

Post by zepearl »

Anthony-G wrote: post #215
simonjgreen wrote: post #145

Earlier quoted context omitted.

I would go as far as to say "hardware raid" these days is limiting, expensive, and less performant that can be achieved with software RAID.

I’m in the process of setting up a home server after buying a pair of matching 3TB Western Digital “Red” disks. I plan on installing them in a HPE ProLiant MicroServer G7 Server / HP Micro G7 N40L that I was gifted a couple of years ago. Even though it comes with a hardware RAID, I was considering setting up RAID 1 using Linux software RAID. However, according to the Linux Raid Wiki ¹, Hardware RAID 1 is better than…

> according to the Linux Raid Wiki¹, Hardware RAID 1 is better than Software RAID 1.

Don't take that too much into consideration - the article was last updated in 2007 ( https://raid.wiki.kernel.org/index.php?title=Overview&action... ) so it lacks some details (the same can be said as well for many ZFS-related infos that you might find) => nowadays doublechecking articles related to raid and ZFS is a must.

In my case I bought some HBA (Host Bus Adapter) cards (e.g. LSI SAS 9211-8i), set their BIOS to not do anything special with the HDDs connected to it (to be able to use them as well with other controllers) and used mdadm (earlier) or ZFS (now) to create my RAIDs => it works well, i get max throughput of ~200MiB per disk, I have all fancy features of ZFS without the problem of proprietary stuff related to the controller card :)

Re: Building a Budget Homelab NAS Server

#235

Post by giantrobot »

bcook wrote: post #151
giantrobot wrote: post #133

Earlier quoted context omitted.

I'd be really careful about SMRs in a RAID. You can end up with no end of performance issues. It's all the downsides of a single SMR drive multiplied by however many drives are in the pools.

I think @dhzhzjsbevs meant that 8TB and higher is CMR. A quick google search seems to support that.

You actually have to be careful. There's disk sizes where the manufacturer will do say 8TB CMR and essentially the same drive with different firmware as a 10TB SMR. They'll also have a 10TB CMR model. You have to pay close attention to the model numbers. It's even more of a crapshoot if you shuck drives. You have to carefully research what externals are known to have CMRs.

SMRs are a fucking blight.

Re: Building a Budget Homelab NAS Server

#236

Post by mrb »

pronoiac wrote: post #139

I'm fairly happy with my 4-bay Synology NAS. When I last looked at ZFS, it seemed that piecemeal upgrades - like, upgrade a 4TB drive to 8TB, and get more available space - wouldn't work in ZFS, but it would in SHR, at least if you had more than 2 drives. Having scheduled checks is a good idea: I have weekly short SMART tests, monthly long SMART tests, and quarterly data scrubs. The TinyPilot device looks nifty - it'…

Oh, my, you are right, the TinyPilot seems awesome! I see it was developed by the author of this ZFS NAS server blog post. I just ordered one to play with :)

Re: Building a Budget Homelab NAS Server

#237

Post by pronoiac »

mrb wrote: post #236
pronoiac wrote: post #139

I'm fairly happy with my 4-bay Synology NAS. When I last looked at ZFS, it seemed that piecemeal upgrades - like, upgrade a 4TB drive to 8TB, and get more available space - wouldn't work in ZFS, but it would in SHR, at least if you had more than 2 drives. Having scheduled checks is a good idea: I have weekly short SMART tests, monthly long SMART tests, and quarterly data scrubs. The TinyPilot device looks nifty - it'…

Oh, my, you are right, the TinyPilot seems awesome! I see it was developed by the author of this ZFS NAS server blog post. I just ordered one to play with :)

Note that The TinyPilot hit the front page with its own post: https://news.ycombinator.com/item?id=31549368

Re: Building a Budget Homelab NAS Server

#238

Post by semi-extrinsic »

axytol wrote: post #226
semi-extrinsic wrote: post #211

Earlier quoted context omitted.

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end. Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

Can you share what your experience with implementing IPoIB with used gear was? I'm asking mainly because I actually got interested recently with such setups however I got rather discouraged by the driver support. As an example here is the driver page for Mellanox, now owned by Nvidia, since they are a major Infiniband equipment supplier: https://network.nvidia.com/products/infiniband-drivers/linux... It seems that so…

Yes, if you want to use the officially supported driver for ConnectX-3 (mlx4_xxx kernel modules, LTS release of v4.9 available from Nvidia's page), you need to go with something like Ubuntu 20.04 LTS (which should be good until at least end of 2025). However, the latest Mellanox drivers (mlx5_xxx kernel modules) work just fine with the ConnectX-3, at least for basic functionality.

I've not actually used IPoIB on such gear myself, but we have been working quite a bit on reusing old/ancient HPC clusters with IB adapters, and you can generally make things work if you spend enough time on trial and error and you are not afraid of compiling code with complicated dependencies. As long as you can get the IB stuff talking, and the driver is using OFED, the IPoIB part should Just Work.

It is always going to be an adventure working with used gear. But HPC has such a high decommissioning tempo and low resale value that there will always be quite a few other enthusiasts toying about.

Re: Building a Budget Homelab NAS Server

#239

Post by mvanbaak »

GekkePrutser wrote: post #184
AviationAtom wrote: post #126

Earlier quoted context omitted.

ZFS on Root just sounds like pain to me. I opt for MD RAID on root and then ZFS my other volumes. I would also say Ubuntu is probably the better choice for Linux ZFS, as CentOS seems to be lacking good support.

ZFS on root is really amazing on FreeBSD and the advantage is that you can snapshot your boot drive.

have a look at Boot Environments. It really is amazing.

Re: Building a Budget Homelab NAS Server

#240

Post by semi-extrinsic »

walrus01 wrote: post #213
semi-extrinsic wrote: post #211

Earlier quoted context omitted.

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end. Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

for home use I'd highly recommend sticking with just 10GbE because you're not locking yourself into a dead-end solution of used weird previous gen infiniband stuff. if you get a $200 switch with a few 10GbE interfaces in it you can easily expand things in the future by trunking vlans to another newer 10GbE capable switch, or connecting to a switch that has multi-gig copper ports for access ports to 2.5/5GBaseT capabl…

You make several good points that I agree with. This is not an easy setup for the inexperienced.

I'll just clarify that I meant you can get a 10m fiber optic cable with two transceivers for $40.

Post reply on HN