161–164 of 164 posts
too bad there was no apple moment (e.g. with thunderbolt) where a third-party was allowed to make it more widespread.
I use two Intel 905 SSDs as mirrored cache devices for ZFS.
Why would you mirror a cache device?
Examples of auxiliary devices where you want redundancy: there can be a write cache (the ZFS Intent Log, or ZIL). You can also dedicate a fast device for hot items like the deduplication tables, or a dedicated device for tiny files where data can be stored directly in the directory data, rather than allocating a separate data block.
Earlier quoted context omitted.
I’ve been considering buying 8x64g models and setting them as equal priority swap disks (to mitigate the low throughput) for this exact reason.
Can confirm doing so is awesome. Get some slightly bigger ones and partition them for additional use as zil. They're extremely satisfying to use, and depressing to remember that we'll never see their like again.
Earlier quoted context omitted.
Can confirm doing so is awesome. Get some slightly bigger ones and partition them for additional use as zil. They're extremely satisfying to use, and depressing to remember that we'll never see their like again.
Do you have any more details? This is such a niche idea, that I’d be buying blind.
Two of U.2x4 to PCIe x16 riser cards, one loaded with 960GB Intel-branded Optanes, one with 1.5TB IBM-branded. PCIe bifurcation is set up in the BIOS to let them all come up properly, where they just show as regular NVMe. Riser cards like this can easily be substituted for PCIe to SAS/Oculink to U.2 cables, if that would be more accommodating to your chassis.
Once they all come up, partition them for your preferred split of swap and ZFS special. Swap should have them all mounted with the same priority and discard=pages, I also recommend setting up zswap (not zram swap) with lz4 as an additional layer of fast, evictable memory pool, as well as `vm.overcommit_memory=2` and `vm.swappiness=150`. This will effectively give you really good memory tiering for workloads and file cache.
When adding the other partitions to ZFS, use `-ashift=12 special mirror dev dev special mirror dev dev ...`. ZFS special covers all metadata, the intent log (sorta write cache), and optionally small files. I like to set it up so With some extra NVMe/Optane allocated to regular ZFS read cache, and all my spinning-rust data VDEVs also as RAID10, it's almost like having the whole array in memory, or at least on fast flash. Eliminating metadata from your drives seeking and letting them be written nearly instantly with Optane does wonderful things for spinning rust :)