Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

github.com

Post reply on HN
Print view View original

11–20 of 78 posts

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#11

Post by prmoustache »

throwaway17_17 wrote: post #5
blobdole wrote: post #2

Beautiful. Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years... But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.

I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives. I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.

> My partner complains about me closing apps on my phone consistently.

I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#12

Post by snailmailman »

prmoustache wrote: post #11
throwaway17_17 wrote: post #5

Earlier quoted context omitted.

I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives. I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.

> My partner complains about me closing apps on my phone consistently. I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?

At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that.

Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#14

Post by gertop »

AnonHP wrote: post #10

I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes. > Fragmentation and extent allocation were adding measurable variance, even on NVMe, Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort o…

SSD still have slower reads when the data isn't sequential. Not slow enough to matter, especially with extents meaning the data is probably in a handful of locations, not 10000, but it's absolutely measurable if your blocks are small enough (<1MB typically, though disk benchmarks usually use 64KB random reads and sometimes 4KB).

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#15

Post by Joel_Mckay »

AnonHP wrote: post #10

I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes. > Fragmentation and extent allocation were adding measurable variance, even on NVMe, Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort o…

>defragmentation won’t achieve much

Indeed, most Linux setups supporting trim, already defer these operations to a weekly schedule to reduce wear, and most fs will optimize in 10MB or 25MB chunks given unlike HDD... the SSD seeks are nearly constant time. Logging fs like f2fs, are content aware so will auto re-locate hot and cold (rarely modified) file types, and despite the log-structure... on an SSD performance losses are often surprisingly negligible.

Most modern NVMe with dram cache and SLC buffer areas also defer committing pages to low-endurance flash areas. And most kernel tweakers will set swapiness to 1 on SSD/NVMe machines to try to keep stuff buffered in dram as long as reasonably possible. It is a space-time tradeoff that can boost a desktop machine performance especially with preload daemon active.

If people want ludicrous speed... than just run ext4 with a separate 128GB journal NVMe drive on a split PCIe x4 bus.

Defrag on most modern drives usually just fills these buffer areas full, and things grind to the slowest i/o choke point. =3

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#16

Post by Joel_Mckay »

gertop wrote: post #14
AnonHP wrote: post #10

I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes. > Fragmentation and extent allocation were adding measurable variance, even on NVMe, Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort o…

SSD still have slower reads when the data isn't sequential. Not slow enough to matter, especially with extents meaning the data is probably in a handful of locations, not 10000, but it's absolutely measurable if your blocks are small enough (<1MB typically, though disk benchmarks usually use 64KB random reads and sometimes 4KB).

Except most SSD also don't store pages sequentially internally (hidden flash wear leveling is even in microsd cards now), and the dram/SLC buffer areas are finite especially on low end budget hardware. Don't defrag your SSD dude. =3

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#17

Post by toast0 »

gertop wrote: post #14
AnonHP wrote: post #10

I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes. > Fragmentation and extent allocation were adding measurable variance, even on NVMe, Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort o…

SSD still have slower reads when the data isn't sequential. Not slow enough to matter, especially with extents meaning the data is probably in a handful of locations, not 10000, but it's absolutely measurable if your blocks are small enough (<1MB typically, though disk benchmarks usually use 64KB random reads and sometimes 4KB).

It's more complex than that... Sequential reads are faster than random reads across most mediums (including system ram), but if there's enough prefetching, you might get better throughput on an SSD with the data fragmented than contiguous because SSDs can often read from multiple regions in parallel and contiguous data might not allow for that.

If you're at the point where you're optimizing for this, you've got some really high performance requirements though. And you'll have to do your own testing, because rules of thumb won't do.

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#18

Post by charcircuit »

snailmailman wrote: post #12
prmoustache wrote: post #11

Earlier quoted context omitted.

> My partner complains about me closing apps on my phone consistently. I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?

At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that. Despite this, many of my relatives have somehow learned this habit of opening the switcher a…

iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#19

Post by __d »

charcircuit wrote: post #9
socalgal2 wrote: post #6

Earlier quoted context omitted.

Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?

Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.

Is it possible to determine which sectors are physically sequential given remapping for wear-leveling?

Otherwise the claimed defragmentation here is not actually resulting in sequential data.

Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter

#20

Post by toast0 »

bitwize wrote: post #7

This is the most useless thing I've seen on here lately. It's not necessary on hard drives, as the page cache did a decent job of improving disk access even in the 90s, let alone with today's fast HDDs. And if you're on an SSD... it'll just burn that out faster. I love it!

Page caches are nice, but you would see meaningful reduction in load times for the OS and applications by getting data in order on spinning disks. Especially games that used most of your ram and loaded reasonably large level files.

fat seemed particularly prone to spreading files into very small chunks across the disk surface; it's not so big of a deal when fragmented files are made up of large contiguous chunks.

Post reply on HN