Pointless and harmful to flash storage. SSD sector numbers are virtual anyway. Its like defragging emails in your inbox.
Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
51–60 of 78 posts
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#52Earlier quoted context omitted.
Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
[flagged]
SSDs often interface with their storage in different (larger) granularities than the OS meaning that even though the data is effectively more random at large scale due to wear leveling there are still benefits to keeping blocks that are sequential in your file sequential in the filesystem also. As well as a potential read boost, the larger blocks introduce a read-before-write issue that can allow multiple close updates.
The difference is very small when compared to the differences between solid state and spinning rust based solutions, but definitely measurable, particularly on drives with DRAM cache, and in some cases human-noticable.
Actually read the article and you'll see the author specifically talks about this begin part of why this side project started (though I suspect nostalgia is the true driving force as there are more efficient ways of addressing that matter!).
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#53Earlier quoted context omitted.
At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that. Despite this, many of my relatives have somehow learned this habit of opening the switcher a…
Android has always been quite vigilant and is prompt to kill off any app at the MFA login screen while I am consulting the Oracle of Authenticator in another task. Thanks Android! That being said, I dislike having 3 dozen apps open because I simply can’t quickly find and switch to the ones I’m using. So when I enter a new situation, I swipe all the way left, and Clear All apps, to start over with a tabula rasa .
The app I use is gone from the app store, but like Spotlight or the Windows Start Menu > type-to-search (before it got utterly shittified with web search/does it have AI now?), I have an Android app called App Search+ on the bottom left of my Android home screen.
Open it, and it opens a list of recently searched for apps. The focus is on the top search bar (and the keyboard has also popped up), and I can type to find any app I have installed.
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#54Earlier quoted context omitted.
[flagged]
SSDs often interface with their storage in different (larger) granularities than the OS meaning that even though the data is effectively more random at large scale due to wear leveling there are still benefits to keeping blocks that are sequential in your file sequential in the filesystem also. As well as a potential read boost, the larger blocks introduce a read-before-write issue that can allow multiple close updat…
I'm pretty sure you don't understand how threads in discussion works.
But, hey. Atleast you can write ~1000 chars with absolutelly no substance fo prove that you know nothing about the topic. What a life to live.
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#55Earlier quoted context omitted.
only HDD drives I use are in a ZFS pool which has a very usable fragmentation value from zpool.
In case you (or others aren’t aware) the FRAG value ZFS shows, is an arbitrary calculation to help give a value to free space fragmentation. The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start findin…
Wow! I did not know that, thank you for the info.
Indeed torrents were the most obvious cause of the fragmentation I saw.
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#56Not to make everything about AI, but.. The current analogy I am using when vibe coding is it's like defragging a hard drive in terms of time wasted. Years of sitting there as the bar crept closer to 99 in the vain hope I could turn the computer off and go to bed, just to have to sit there for another hour or so until finished. So many late nights wasted doing, I'm not sure what. And now I'm sat there waiting for the…
poetic :)
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#57Beautiful. Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years... But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all.
This is on a slow device but it might be worse on a fast one as the extra allocation of small extents start to hit harder on the host side:
(fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#58I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes. > Fragmentation and extent allocation were adding measurable variance, even on NVMe, Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort o…
Yes: I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs so this is not helping at all.
(fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#59Not to make everything about AI, but.. The current analogy I am using when vibe coding is it's like defragging a hard drive in terms of time wasted. Years of sitting there as the bar crept closer to 99 in the vain hope I could turn the computer off and go to bed, just to have to sit there for another hour or so until finished. So many late nights wasted doing, I'm not sure what. And now I'm sat there waiting for the…
What actually triggered me to do that was to be able to test repeatably my other open source project in rust were the compile target directory can go to hundreds of gigabytes of files and at the same time I need to benchmark it on large linear slabs we do create for memory mapped files and I definitely noted the impact. I can only guess but non-optimal extent length create a repeated round trip to the metadata in main memory instead of "linear reads" (as consecutive logical numbers) + maybe the fact that those SSD are probably super smart and prefetch the next sectors? (see the jitter numbers I get even on a slow device in other comments)
Re: Show HN: I missed the moving blocks, so I built a real Linux disk defragmenter
#60Neat! I have several questions here. First of all, why does Linux not need defragmentation? Second, is it actually true that it doesn't need it? The readme implies that this defragmenter (for Linux) produces actual benefits. Third, it says the benefits exist even on NVMe, i.e. on SSDs? I thought defragging was relevant for spinny disks only? Thanks
Mostly, they try to avoid it by allocating extents instead of blocks (so, they look for a contiguous space to store most files) and by delaying allocation to accommodate for a growing files.
My observation is that it is not completely true. I constantly have a close to full partition in ext4 and there is no secrets when there is no space the filesystem WILL create a bazillion of small extents because, it has no physical choice! and everytime you have a non maxed out extent, it will create an overhead I don't see how magically you can avoid that. And it is not like a few, on my main working partition, extends of a few KB by the thousands per files are created instead of the optimal size of 128MB. If you just use defrag in read mode on it, it is red from top to bottom, you cannot find a block of few MB with no fragmented file in it.