Live data from Hacker News

Hyperspace

hypercritical.co

141–150 of 503 posts

Re: Hyperspace

#141

> Hyperspace can’t be installed on “Macintosh HD” because macOS version 15 or later is required. macOS 15 was released in September 2024, this feels far too soon to deprecate older versions.

Can it really be seen as deprecating an old version when it’s a brand new app?

Re: Hyperspace

#142
I have file A that's in two places and I run this.

I modify A_0. Does this modify A_1 as well or just kind of reify the new state of A_0 while leaving A_1 untouched?

Re: Hyperspace

#143
post #123
post #60

I did this with two scripts - one that produces and cached sha1 sums of files, and another that consumes the output of the first (or any of the *sum progs) and produces stats about duplicate files, with options to delete or hard-link them.

I wonder how any comments about hard links will be in these comments by people misunderstanding what this app does.

if file is not going to be modified (in the low-level sense - open("w") on the filename; as opposed to rename-and-create-new), then reflinks (what this app does) and hardlinks act somewhat identically.

For example if you have multiple node_modules, or app installs, or source photos/videos (ones you don't edit), or music archives, then hardlinks work just fine.

Re: Hyperspace

#145

https://github.com/pkolaczk/fclones can do the same thing, and it's perfectly free and open source. terminal based though

Hyperspace said I can save 10GB.

But then I ran this command and saved over 20GB:

  brew install fclones
  cd ~
  fclones group . | fclones dedupe
I've used fclones before in the default mode (create hard links) but this is the first time I've run it at the top level of my home folder, in dedupe mode (i.e. using APFS clones). Fingers crossed it didn't wreck anything.

Re: Hyperspace

#146
post #88
post #48

> There is no way for Hyperspace to cooperate with all other applications and macOS itself to coordinate a “safe” time for those files to be replaced, nor is there a way for Hyperspace for forcibly take exclusive control of those files. This got me wondering why the filesystem itself doesn't run a similar kind of deduplication process in the background. Presumably, it is at a level of abstraction where it could safel…

On ZFS it consumes a lot of RAM. In part I think this is because ZFS does it on the block level, and has to keep track of a lot of blocks to compare against when a new one is written out. It might be easier on resources if implemented on the file level. Not sure if the implementation would be simpler or more complex. It might also be a little unintuitive that modifying one byte of a large file would result in a lot d…

Is there a FS that keeps only diffs in clone files? It would be neat

Re: Hyperspace

#147

Earlier quoted context omitted.

My understanding is that it is a copy-on-write clone, not a hard link. [1] > Q: Are clone files the same thing as symbolic links or hard links? > A: No. Symbolic links ("symlinks") and hard links are ways to make two entries in the file system that share the same data. This might sound like the same thing as the space-saving clones used by Hyperspace, but there’s one important difference. With symlinks and hard links…

What kind of changes could you make to one clone that would still qualify it as a clone? If there are changes, it's no longer the same file. Even after reading the How It Works[0] link, I'm not groking how it works. Is it making some sort of delta/diff that is applied to the original file? That's not possible for every file format like large media files. I could see that being interesting for text based files, but th…

If I understand correctly, a COW clone references the same contents (just like a hardlink) as long as all the filesystem references are pointing to identical file contents.

Once you open one of the reference handles and modify the contents, the copy-on-write process is invoked by the filesystem, and the underlying data is copied into a new, separate file with your new changes, breaking the link.

Comparing with a hardlink, there is no copy-on-write, so any changes made to the contents when editing the file opened from one reference would also show up if you open the other hardlinks to the same file contents.

Re: Hyperspace

#148
post #49

I gave it a try on my massive folder of NodeJS projects but it only found 1GB of savings on a 8.1GB folder. I then tried again including my user home folder (731K files, 127K folders, 2755 eligible files) to hopefully catch more savings and I only ended up at 1.3GB of savings (300MB more than just what was in the NodeJS folders.) I tried to scan System and Library but it refused to do so because of permission issues.…

pnpm tries to be a drop-in replacement for npm, and dedupes automatically.

More importantly, pnpm installs packages as symlinks, so the deduping is rather more effective. I believe it also tries to mirror the NPM folder structure and style of deduping as well, but if you have two of the same package installed anywhere on your system, pnpm will only need to download and save one copy of that package.

Re: Hyperspace

#149
What algorithm does the application use to figure out if two files are identical? There's a lot of interesting algorithms out there. Hashes, bit by bit comparison etc. But these techniques have their own disadvantages. What is the best way to do this for a large amount of files?

Re: Hyperspace

#150
post #141

> Hyperspace can’t be installed on “Macintosh HD” because macOS version 15 or later is required. macOS 15 was released in September 2024, this feels far too soon to deprecate older versions.

Can it really be seen as deprecating an old version when it’s a brand new app?

+1. He's not taking anything away because you never had it.
Post reply on HN