Just want to mention: Apple ships a modified version of the copy command (good old cp) that supports the ability to use the cloning feature of APFS by using the -c flag.
Hyperspace
401–410 of 503 posts
Re: Hyperspace
#402Downloaded. Ran it. Tells me "900" files can be cleaned. No summary, no list. But I was at least asked to buy the app. Why would I buy the app if I have no idea if it'll help?
[deleted]
Re: Hyperspace
#403Its interesting how Linux tools are all free when even trivial mac tools are being sold. Nothing against someone trying to monetize but the linux culture sure is nice!
There might be a difference in robustness. There's a monetary consequence to this developer for getting it wrong.
The linux tools can't get it wrong, the kernel checks that the files submitted for deduplication are actually identical.
Re: Hyperspace
#404Requires macOS 15.0 or later. – Oh god, this is so stupid and most irritating thing about macOS "Application development". It is really unfair to call it "software" it is more like "glued to recent version of OS ware", meanwhile I can still run .exe compiled in 2006, and with wine even on mac or linux.
I would also have appreciated a version that's compatible with a non-latest macOS release.
Then again, this app was written with SwiftUI, which hasn't received some handy features before macOS 12 and is still way behind AppKit.
When I see an app that's not compatible with the second most recent macOS, I assume the dev either didn't know better or they were too lazy to write workarounds / shims for the latest-and-greatest shiny stuff.
Re: Hyperspace
#405Earlier quoted context omitted.
To make dedup[0] fast, I use a tree with device id, size, first byte, last byte, and finally SHA-256. Each of those is only used if there is a collision to avoid as many reads as possible. dedup doesn’t do a full file compare, because if you’ve found a file with the same size, first and last bytes, and SHA-256 you’ve also probably won the lottery several times over and can afford data recovery. This is the default fo…
How much time is saved by not comparing full file contents? Given that this is a tool some people will only run occasionally, having it take 30 seconds instead of 15 is a small price to pay for ensuring it doesn't treat two differing files as equal.
Same size, same first and last bytes, and same SHA-256.
…and you’re not worried about shark attacks, are you?
Re: Hyperspace
#406My personal head canon is that Steve Jobs personally cancelled ZFS in OSX because Jonathan Schwartz prematurely announced it.
Re: Hyperspace
#407> There is no way for Hyperspace to cooperate with all other applications and macOS itself to coordinate a “safe” time for those files to be replaced, nor is there a way for Hyperspace for forcibly take exclusive control of those files. This got me wondering why the filesystem itself doesn't run a similar kind of deduplication process in the background. Presumably, it is at a level of abstraction where it could safel…
Disk Utility.app manages to keep the OS running while make the disk exclusive-access.. I wonder how it does that.
It doesn't? Freeze when commiting operations.
Re: Hyperspace
#408Earlier quoted context omitted.
Replying to myself now that I've had a chance to try the scan, but not the deduplication. I work with disc images, program binaries, intermediate representations in a workspace that's 7.6G. A few notes: * By default it doesn't scan everything. It ignores all files but those in an allow list. The way the allow list is structured, it seems like Hyperspace needs to understand the content of a file. As an end user, I hav…
I'm a little surprised that folks here are investing so much time into this app. It's closed source, only available for a non-obious amount, time-limited or subscription-based and lots of details of how it works are missing. With a FOSS project this would have been expected, but with a ShareWare-style model? Idk..
John has reiterated multiple times on his podcast that he doesn't want to deal with thousands of support requests when making his apps open source and free. All his apps are personal itches he scratched and he sells them not to make a profit but to make the barrier of entry high enough to make user feedback manageable.
Re: Hyperspace
#409Earlier quoted context omitted.
On ZFS it consumes a lot of RAM. In part I think this is because ZFS does it on the block level, and has to keep track of a lot of blocks to compare against when a new one is written out. It might be easier on resources if implemented on the file level. Not sure if the implementation would be simpler or more complex. It might also be a little unintuitive that modifying one byte of a large file would result in a lot d…
Is there a FS that keeps only diffs in clone files? It would be neat
With extent-based filesystems you can clone extents and then overwrite one extent and only that becomes unshared.
Re: Hyperspace
#410> There is no way for Hyperspace to cooperate with all other applications and macOS itself to coordinate a “safe” time for those files to be replaced, nor is there a way for Hyperspace for forcibly take exclusive control of those files. This got me wondering why the filesystem itself doesn't run a similar kind of deduplication process in the background. Presumably, it is at a level of abstraction where it could safel…
On ZFS it consumes a lot of RAM. In part I think this is because ZFS does it on the block level, and has to keep track of a lot of blocks to compare against when a new one is written out. It might be easier on resources if implemented on the file level. Not sure if the implementation would be simpler or more complex. It might also be a little unintuitive that modifying one byte of a large file would result in a lot d…
That's a ZFS online dedup limitation. I think xfs and btrfs are better prior art here since they use extent-based deduplication and can do offline dedup which means they don't have have to keep it in memory and the on-disk metadata is smaller too.