261–270 of 503 posts
I gave it a try on my massive folder of NodeJS projects but it only found 1GB of savings on a 8.1GB folder. I then tried again including my user home folder (731K files, 127K folders, 2755 eligible files) to hopefully catch more savings and I only ended up at 1.3GB of savings (300MB more than just what was in the NodeJS folders.) I tried to scan System and Library but it refused to do so because of permission issues.…
Earlier quoted context omitted.
APFS is a copy on write filesystem if you use the right APIs, so it does what you describe but only for entire files. I believe as soon as you change a single bite you get a complete copy that’s your own. And that’s how this program works. It finds perfect duplicates and then effectively deletes and replaces them with a copy of the existing file so in the background there’s only one copy of the bits on the disk.
I suppose this means that you could find yourself unexpectedly out of disk space in unintuitive ways, if you're only trying to change one byte in a cloned file but there isn't enough space to copy its entire contents?
As far as I understand, it works like a reflink feature in the modern linux FSs. If so, thats really cool, and thats also a bit better than the zfs's snapshots. Iam newbie on macos, but it looks amazing
> There is no way for Hyperspace to cooperate with all other applications and macOS itself to coordinate a “safe” time for those files to be replaced, nor is there a way for Hyperspace for forcibly take exclusive control of those files. This got me wondering why the filesystem itself doesn't run a similar kind of deduplication process in the background. Presumably, it is at a level of abstraction where it could safel…
If Apple is anything like where I work, there's probably a three-year-old bug ticket in their system about it and no real mandate from upper management to allocate resources for it.
CLI tool to find duplicate files unbelievably quickly: https://github.com/twpayne/find-duplicates
Earlier quoted context omitted.
Swift 6 is not the problem. It's backward compatible. The problem is SwiftUI. It's very new, still barely usable on the Mac, but they are adding lots of new features every macOS release. If you want to support older versions of macOS you can't use the nice stuff they just released. Eg. pointerStyle() is a brand new macOS 15 API that is very useful.
It's been a while since I last looked at SwiftUI on mac, Is it really still that bad ?
They are working on it, and making it better every year. I've started using it for small projects and it's pretty neat how fast you can work with it -- but not everything can be done yet.
Since they are still adding pretty basic stuff every year, it really hurts if you target older versions. AppKit is so mature that for most people it doesn't matter if you can't use new features introduced in the last 3 years. For SwiftUI it still makes a big difference.
I have to confess: it miffs me that a utility that would normally fly completely under the radar is likely to make the creator thousands of dollars just because he runs a popular podcast. (Am I jealous? Oh yes. But only because I tried to sell similar apps in the past and could barely get any downloads no matter how much I marketed them. Selling software without an existing network seems nigh-on impossible these days…
The author is a "household" name in the macOS / Apple scene for a long time even before the podcast. If someone is spending all their life blogging about all things Apple on outlets like ArsTechnica and is consistently putting out new content on podcasts for decades they will naturally have a better distribution.
How many years did you spend on building up your marketing and distribution reach?
I have to confess: it miffs me that a utility that would normally fly completely under the radar is likely to make the creator thousands of dollars just because he runs a popular podcast. (Am I jealous? Oh yes. But only because I tried to sell similar apps in the past and could barely get any downloads no matter how much I marketed them. Selling software without an existing network seems nigh-on impossible these days…
I can understand your criticism as it's easy to arrive at that conclusion (Also a common occurrence when levelsio launches a new product, as his Twitter following is large) but it's also not fair to discount it as "just because he runs a popular podcast". The author is a "household" name in the macOS / Apple scene for a long time even before the podcast. If someone is spending all their life blogging about all things…
Earlier quoted context omitted.
A ~20 y.o. account with perhaps hundreds of devices in history across different continents and countries along with family sharing. Every time I need to purchase something via Apple, it becomes a quest. Enter password, validate card, welcome to endless login loop. Reboot. Click purchase, enter password, confirm OTP on another device, then nothing happens, purchase button is active, clicks ignored. Reboot. Click "Get"…
I would never dismiss such a complaint with a glib "works for me". And yet, your experience is so utterly, completely different from mine that I have to think something's busted in your account somewhere. I've had an account for about as long, with family sharing and all the rest. I never, ever, have anywhere near that level of difficulty. For me it works as documented: I click "Get", it asks for Face ID to confirm i…
I can't complain though because I have this account for like couple decades and loosing that account would be painful. Apple did ban my account twice on the grounds that it's a US account while I'm not physically located there. I was able to revert the ban by explaining that I've got a US legal entity (account, banking card, etc) and thus I beg to continue using it. Not taking chances for the third time, so I silently endure.
Earlier quoted context omitted.
In order to check if a file is a duplicate of another, you need to check it against _every other possible file_. You need some kind of "lookup key". If we took the first 1024 bytes of each file as the lookup key, then our key size would be 1024 bytes. If you have 1 million files on your disk, then that's 128MB of ram just to store all the keys. That's not a big deal these days, but it's also annoying if you have a bu…
I understand the concept. My main point is that it's probably not a huge advantage to store hashes of the first 1KB, which requires CPU to calculate, over just the raw bytes, which requires storage. There's a tradeoff either way. I don't think it would be far more efficient to do hash the entire contents though. If you have a million files storing a terabyte of data, the 2 stage comparison would read at most 1GB (1 m…
... okay, so as long as you always feed chunks of data into your hash in the same deterministic order, it doesn't matter for the sake of correctness what that order is or even if you process some bytes multiple times. You could hash the first 1kB, then the second-through-last disk blocks, then the entire first disk block again (double-hashing the first 1kB) and it would still tell you whether two files are identical.
If you're reading from an SSD and seek times don't matter, it's in fact probable that on average a lot of files are going to differ near the start and end (file formats with a header and/or footer) more than in the middle, so maybe a good strategy is to use the first 32k and the last 32k, and then if they're still identical, continue with the middle blocks.
In memory, per-file, you can keep something like
- the length
- h(block[0:4])
- h(block[0:4] | block[-5:])
- h(block[0:4] | block[-5:] | block[4:32])
- h(block[0:4] | block[-5:] | block[4:128])
- ...
- h(block[0:4] | block[-5:] | block[4:])
A further refinement would be to have per-file-format hashing strategies... but then hashes wouldn't be comparable between different formats, so if you had 1M pngs, 1M zips, and 1M png-but-also-zip quine files, it gets weird. Probably not worth it to go down this road.
Earlier quoted context omitted.
It's called copy-on-write because when you modify A_0, it'll make a copy of the file if you write to it but not A_1. https://en.wikipedia.org/wiki/Copy-on-write#In_computer_stor...
Which means if you actually edited those files, you might fill up your HD much more quickly than you expected. But if you have the same 500MB of node_modules in each of your dozen projects, this might actually durably save some space.
I'm not sure if this is what you intended, but just to be sure: writing changes to a cloned file doesn't immediately duplicate the entire file again in order to write those changes — they're actually written out-of-line, and the identical blocks are only stored once. From [the docs](^1) posted in a sibling comment:
> Modifications to the data are written elsewhere, and both files continue to share the unmodified blocks. You can use this behavior, for example, to reduce storage space required for document revisions and copies. The figure below shows a file named “My file” and its copy “My file copy” that have two blocks in common and one block that varies between them. On file systems like HFS Plus, they’d each need three on-disk blocks, but on an Apple File System volume, the two common blocks are shared.
[^1]: https://developer.apple.com/documentation/foundation/file_sy...