Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If Scrubs Hurt, Your ZFS Design Is Broken

klarasystems.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#11

Post by secabeen »

zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

Yeah, the only place I use SLOG is with Optane devices in front of SATA/SAS flash. Anything that needs the speed of SLOG shouldn't be on rust anyways.

I would be in a different world if my system was backing banking or other loss critical data. For all my workloads, losing 5 seconds of writes in a crash is NBD.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#12

Post by toast0 »

zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

> 1. From my own experience, most people who use a ZIL for faster sync writes are wasting space. You only need five seconds of peak throughput (for a 10Gb connection that's ~5 GB). Create a pair of 25 GB partitions for the ZIL rather than dedicating entire devices.

You might benefit from the same technique for a "special" vdev... If you run an all spinners array, but the metadata is on a small partition, you might get better seek times during metadata scans, since everything will be closer together? And/or if you use raidz for your regular vdev, but mirror for the metadata, you'll have more capacity for metadata reads.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#13

Post by nubinetwork »

zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

> Even my 24 TB NAS takes 24+ hours with no other activity

Your NAS sounds pretty slow, it shouldn't take an hour to scrub 1tb of data (on average)

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#14

Post by [deleted] »

mindslight wrote: post #4

Oh sure, just buy more disks. Who do you think I am, that guy on the Monopoly box? I stocked up on a few shuckable externals before the start of the second Grump catastrophe. $280/24TB (Exos inside). I'm finally getting around to setting up a pool, thinking 4+1 is kind of poor redundancy and wondering about adding another disk. The things are up to over $1,000 each, in the husk or not. Fucking hell. It makes me want…

[deleted]

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#15

Post by ls612 »

nubinetwork wrote: post #13
zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

> Even my 24 TB NAS takes 24+ hours with no other activity Your NAS sounds pretty slow, it shouldn't take an hour to scrub 1tb of data (on average)

I have 3x22 TB drives in a RaidZ1 pool with roughly 20TB of data and a scrub takes 18 hours. This sounds about right to me tbh.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#16

Post by myrandomcomment »

ls612 wrote: post #15
nubinetwork wrote: post #13

Earlier quoted context omitted.

> Even my 24 TB NAS takes 24+ hours with no other activity Your NAS sounds pretty slow, it shouldn't take an hour to scrub 1tb of data (on average)

I have 3x22 TB drives in a RaidZ1 pool with roughly 20TB of data and a scrub takes 18 hours. This sounds about right to me tbh.

VDEVs:

Data VDEVs - 4 x MIRROR, 2 wide, 12.73TB; Log VDEVs - 1 x DISK, 14.91TB, 1 wide; Cache VDEVs - 1 x 447.13 GiB

HW:

8 x WDC_WD140EFGX-68B0GN08; 2 x Micron_5300_MTFDDAK480TDS

Total space is 50.81TB of which 24.16TB is used. Frag on pool is reported at 21%.

Last Scrub took 11 hours 45 minutes 50 seconds.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#17

Post by ls612 »

myrandomcomment wrote: post #16
ls612 wrote: post #15

Earlier quoted context omitted.

I have 3x22 TB drives in a RaidZ1 pool with roughly 20TB of data and a scrub takes 18 hours. This sounds about right to me tbh.

VDEVs: Data VDEVs - 4 x MIRROR, 2 wide, 12.73TB; Log VDEVs - 1 x DISK, 14.91TB, 1 wide; Cache VDEVs - 1 x 447.13 GiB HW: 8 x WDC_WD140EFGX-68B0GN08; 2 x Micron_5300_MTFDDAK480TDS Total space is 50.81TB of which 24.16TB is used. Frag on pool is reported at 21%. Last Scrub took 11 hours 45 minutes 50 seconds.

Mine are WD Red Pros. I also have a 2TB WD Black SSD as a cache VDEV for the pool.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#18

Post by zenoprax »

toast0 wrote: post #12
zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

> 1. From my own experience, most people who use a ZIL for faster sync writes are wasting space. You only need five seconds of peak throughput (for a 10Gb connection that's ~5 GB). Create a pair of 25 GB partitions for the ZIL rather than dedicating entire devices. You might benefit from the same technique for a "special" vdev... If you run an all spinners array, but the metadata is on a small partition, you might ge…

That's an interesting idea. Reminds me of how people would put their swap partitions at the end for faster speeds. If I did an N-way mirror I get could attach/detach as needed and get much more performance on reads.

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#19

Post by zenoprax »

ls612 wrote: post #15
nubinetwork wrote: post #13

Earlier quoted context omitted.

> Even my 24 TB NAS takes 24+ hours with no other activity Your NAS sounds pretty slow, it shouldn't take an hour to scrub 1tb of data (on average)

I have 3x22 TB drives in a RaidZ1 pool with roughly 20TB of data and a scrub takes 18 hours. This sounds about right to me tbh.

I'm running five 8TB in a Z2 so I think that slows me down a bit more especially compared to people running mirrors. I'm going to kick a scrub off now and see if I'm misremembering.

---

No need, looks like it ran a few weeks ago:

scrub repaired 0B in 12:43:52 with 0 errors on Mon Jul 6 20:58:58 2026

Re: If Scrubs Hurt, Your ZFS Design Is Broken

#20

Post by zenoprax »

nubinetwork wrote: post #13
zenoprax wrote: post #5

I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides. 1. From my own experience, most people who use a ZIL for faster sync w…

> Even my 24 TB NAS takes 24+ hours with no other activity Your NAS sounds pretty slow, it shouldn't take an hour to scrub 1tb of data (on average)

I was off by 2x - last scrub took 12 hours.
Post reply on HN