121–130 of 316 posts
Earlier quoted context omitted.
The active pcie switches are a lot more expensive than the passive splitters that only work on systems with pcie bifurcation.
For a large scale storage system (more NVMe devices than PCIe lanes), a switch is mandatory. Or a “RAID” card or another switch-like device.
Earlier quoted context omitted.
I see, so I assume the upside is that it's a time saver. Thanks! I personally wen't with samba on Linux and with btrfs. I was wondering if there's something non-obvious in TrueNAS that I'm missing out on. And to my account, I think my upsides are that: - ability to choose the kernel - no need for SSD for base OS since running off of RAM is rather easy on Linux - samba can run in a container thus a bit more control se…
There are two flavors of TrueNAS - Core and Scale. Core is basically a FreeBSD distro and Scale is basically a Linux distro. They're both a base OS with the typical packages anyone would need for a NAS, with sane defaults + a user-friendly web-based management system. The upsides are that it's plug-and-play for anyone who doesn't want to research all the options available and figure out the various pitfalls on their…
> You're running off a RAM disk with no boot drive? What if you have a power outage?
Yes, the server only has the HDDs which contain the NAS data. The server bootloops until it gets an image from the router (ipxe boot). The disk images have systemd scripts which install everything from 0 on each boot. Coincidentally, this means system restart is how I upgrade my software.
> Core supports FreeBSD jails and Scale supports Docker
This clarifies the situation -- TrueNAS seems like an option that I would recommend for anyone who wants a quick OSS NAS setup.
I wonder if it’s possible to back up ZFS (eg, a ZFS server) to btrfs (eg, a synology nas)? I mean backing up the file system (as with ZFS send) not scanning all files (using rsync of restic).
This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…
I would advise using ZFS only with distros that come with it (i.e. Ubuntu, Proxmox), especially if you plan to have your / on it. I wasted too much time on CentOS with ZFS, would not do it again.
A 500W PSU won't necessarily draw more than a 250W PSU, that is merely its maximum sustained load (what the rest of the system asks for) rating. The Bronze 80+ rating is likely part of the problem here, that indicates what the power draw from the wall is compared to what is being provided to your system. Titanium 80+ would net you about 10% reduction in wall power usage. Keep in mind that manufacturers play fast and loose with the certification process and a consumer unit may not actually be what it says on the box, you need to rely on quantitative reviews.
Other than that, spend some time in the firmware settings. Powtop also does a great job at shaving off some watts.
This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…
> I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. I would advise using ZFS only with distros that come with it (i.e. Ubuntu, Proxmox), especially if you plan to have your / on it. I wasted too much time on CentOS with ZFS, would not do it again.
I would also say Ubuntu is probably the better choice for Linux ZFS, as CentOS seems to be lacking good support.
Thinking of doing the same for a similar scale! Thanks for sharing.
Earlier quoted context omitted.
I know about CephFS, but performance was abysmal compared to ZFS for a home server. On a single box with 4-8 drives I didn't come close to saturating a 10G link, which ZFS managed just fine. It was also very complex to manage compare to ZFS, with many different layers to consider. I'm sure it shines in a data center, for which it has been designed. But unless something radical has changed in the last year, it's not f…
The cephfs per-machine redundancy mode is usually the preferred configuration. i.e. usually avoids cramming everything into a single point of failure, buying specialty SAS cards, and poking at live raid arrays to do maintenance. Seen too many people's TrueNAS/FreeNAS installs glitch up over the years to trust the zfs community edition as a sane production choice. ZFS certainly has improved, but Oracle is not generall…
BTRFS seems to be maturing nicely, hopefully we can start using it for these types of workloads in the next few years.
Earlier quoted context omitted.
Also possible with BTRFS. I set this up once for a small business with hourly snapshots during working hours. This way users could just restore older versions of files they accidentally deleted, overwrote or messed up in some other way. Another benefit: Those snapshots were read-only, so they also served as a protection against ransomware.
I don't think BTRFS supports NFSv4 ACLs yet (ie, Windows ACLs are natively supported on ZFS, there is patchset so Linux also supports it but BTRFS obviously has no integration for a patchset that only exists for ZFS). Having NFSv4 ACL access is a huge plus since you can configure permissions natively from windows and have them enforced even on the shell.
Author here. I consider myself an intermediate homelabber and a TrueNAS beginner. I just built my first NAS server, so wrote this to capture everything I wish I'd known at the start. I hope it's helpful for anyone else thinking about building their first NAS server. Any questions or feedback about the post are more than welcome.
RAID-Z1 is something I never consider without a solid backup to restore from and a plan to execute that process at least once in the lifecycle of an array.
If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. The good news is that ZFS will tell you exactly which files you have lost data for and cannot rebuild. If you have those files, you can overwrite them with the backups to get your integrity back.
The reason is, with a total loss of a single disk, any read error on any of the remaining disks is a lost/corrupted file.
For this reason, you need a strong(easily accessible, consistent, current) backup strategy and an acceptance of downtime with Z1.
As for ECC, it's better, but your absolute worse case scenario is that you get a bit flip before the sync and hash happens, and now that bit flipped data is committed to disk and you think it's OK. I prefer ECC to avoid this, but you are still reaping a multitude of benefits from ZFS without ECC.
The only valid rule for RAM and ZFS is that more RAM = more caching of recently read data. Single, or very few user appliances will see little benefit past 8GB even with 100TB unless you happen to be reading the same data over and over. Where ZFS shines is having hundreds of gigabytes of RAM and tens or more concurrent users mostly accessing the same data. That way the vast majority of reads are from RAM and the overall disk IOPS remain mostly idle.
Most of the ZFS RAM myths come from Deduplication, which should be disregarded as a ZFS feature until they allow storing of the DDT on a Optane-like latency device. Even better would be offline deduplication, but I doubt that will be a thing in ZFS this decade.