201–210 of 503 posts
Earlier quoted context omitted.
I'm a bit confused as the Mac App Store says it's over 4 years old.
The 4+ Age rating is like, who can use the app. Not for 3 year olds, apparently.
What algorithm does the application use to figure out if two files are identical? There's a lot of interesting algorithms out there. Hashes, bit by bit comparison etc. But these techniques have their own disadvantages. What is the best way to do this for a large amount of files?
I'd hash the first 1024 bytes of all files, and starts from there is any collision. That way you don't need to hash the whole (large) files, but only those with same hashes.
Also, use the length of the file for a fast check.
Nice, but I'm not getting a subscription for a filesystem utility. Had it been a one-time $5 license, I would have bought it. At the current price, it's literally cheaper to put files in a S3 bucket or outright buy an SSD.
There are several such tools for Linux, and they are free, so maybe just change operating systems.
rmlint -c sh:handler=reflink .
Earlier quoted context omitted.
I think the term to search for is reflink. Btrfs is one example: https://btrfs.readthedocs.io/en/latest/Reflink.html Like with Hyperspace, you would need to use a tool that can identify which files are duplicates, and then convert them into reflinks.
I thought reflink is provided by the underlying FS, and Hyperspace is a dedup tool that finds the duplicates.
You only get CoW on APFS if you copy a file with certain APIs or tools.
If you have a program that does it manually, you copied a duplicate to somewhere on your desk from some other source, or your files already existed on the file system when you converted to APFS because you’ve been carrying them for a long time then you’d have duplicates.
APFS doesn’t look for duplicates at any point. It just keeps track of those that it knows are duplicates because of copy operations.
Nice, but I'm not getting a subscription for a filesystem utility. Had it been a one-time $5 license, I would have bought it. At the current price, it's literally cheaper to put files in a S3 bucket or outright buy an SSD.
I don't understand why a simple, closed source de-dup app is at the top of the front page with 160+ comments? What is so interesting about it? I read the blog and the comments here and I still don't get it.
The developer is popular and APFS cloning is genuinely technically interesting. (no, it's not a symlink)
I don't understand why a simple, closed source de-dup app is at the top of the front page with 160+ comments? What is so interesting about it? I read the blog and the comments here and I still don't get it.
Earlier quoted context omitted.
I don't know exactly what Siracusa is doing here, but I can take an educated guess: For each candidate file, you need some "key" that you can use to check if another candidate file is the same. There can be millions of files so the key needs to be small and quick to generate, but at the same time we don't want any false positives. The obvious answer today is a SHA256 hash of the file's contents; It's very fast, not t…
I think the prob. is not so low. I remember reading here about a person getting a foto of another chat in a chat application, which was using sha in the background. I do not recall all the details, it is improbable, but possible.
https://crypto.stackexchange.com/questions/47809/why-havent-...
Nice, but I'm not getting a subscription for a filesystem utility. Had it been a one-time $5 license, I would have bought it. At the current price, it's literally cheaper to put files in a S3 bucket or outright buy an SSD.
Edit: Apparently both is possible in the end: https://hypercritical.co/hyperspace/#purchase
Nice, but I'm not getting a subscription for a filesystem utility. Had it been a one-time $5 license, I would have bought it. At the current price, it's literally cheaper to put files in a S3 bucket or outright buy an SSD.
There are several such tools for Linux, and they are free, so maybe just change operating systems.