Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hyperspace

hypercritical.co

Post reply on HN
Print view View original

451–460 of 503 posts

Re: Hyperspace

#451

Post by apercu »

egorfine wrote: post #40
apercu wrote: post #25

Earlier quoted context omitted.

[flagged]

A ~20 y.o. account with perhaps hundreds of devices in history across different continents and countries along with family sharing. Every time I need to purchase something via Apple, it becomes a quest. Enter password, validate card, welcome to endless login loop. Reboot. Click purchase, enter password, confirm OTP on another device, then nothing happens, purchase button is active, clicks ignored. Reboot. Click "Get"…

Oh, I was agreeing with you. My 2015 iMac died two weeks ago. Pretty sure it's the SSD I installed when I first got it. And while most of my files are in cloud storage, I also had a series of chained external drives running Time Machine. Guess what? I can't use any apple tools to grab any files because different file system type between that machine and my M1 MB Pro (permissions issues).

I'm going to have to clone the drive, then use terminal to chmod the /usr dirs to extract the files I want (mostly personal music production).

I immediately ordered a Mac mini, but since I didn't want a 256GB drive and 16GB RAM, I'm still waiting for it to arrive from China.

Also, the M1 MB Pro was the most expensive* and worst computer I have ever owned. I wish I had just bought an air. No tactile volume controls. As a musician that is the worst.

(*company I worked for a while in school paid $12k for a Mac 2 FX. Lol.)

Re: Hyperspace

#452

Post by apercu »

egorfine wrote: post #268
kstrauser wrote: post #154

Earlier quoted context omitted.

I would never dismiss such a complaint with a glib "works for me". And yet, your experience is so utterly, completely different from mine that I have to think something's busted in your account somewhere. I've had an account for about as long, with family sharing and all the rest. I never, ever, have anywhere near that level of difficulty. For me it works as documented: I click "Get", it asks for Face ID to confirm i…

Oh yes, it's pretty clear to me that something is wrong on Apple's side specifically with my account. Obviously people are having close to zero friction with Apple's stuff. I can't complain though because I have this account for like couple decades and loosing that account would be painful. Apple did ban my account twice on the grounds that it's a US account while I'm not physically located there. I was able to rever…

> Obviously people are having close to zero friction with Apple's stuff.

I don't believe that to be true, I have been having issues with Apple bugs for the last 7-8 years. Totally unnecessary friction due to features I do not want and do not use.

Re: Hyperspace

#453

Post by jlhawn »

kccqzy wrote: post #438
alwillis wrote: post #401

Just want to mention: Apple ships a modified version of the copy command (good old cp) that supports the ability to use the cloning feature of APFS by using the -c flag.

And in case your cp doesn't support it, you could also do it by invoking Python. Something like `import Foundation; Foundation.NSFileManager.defaultManager().copyItemAtPath_toPath_error_(...)`.

that function name `copyItemAtPath_toPath_error_` sounds like it was ported directly from objective-c!

Re: Hyperspace

#454

Post by pornel »

jonhohle wrote: post #351
MBCook wrote: post #180

Earlier quoted context omitted.

You can start with the size, which is probably really unique. That would likely cut down the search space fast. At that point maybe it’s better to just compare byte by byte? You’ll have to read the whole file to generate the hash and if you just compare the bytes there is no chance of hash collision no matter how small. Plus if you find a difference in bytes 1290 you can just stop there instead of reading the whole t…

To make dedup[0] fast, I use a tree with device id, size, first byte, last byte, and finally SHA-256. Each of those is only used if there is a collision to avoid as many reads as possible. dedup doesn’t do a full file compare, because if you’ve found a file with the same size, first and last bytes, and SHA-256 you’ve also probably won the lottery several times over and can afford data recovery. This is the default fo…

Reading just the first byte is probably wasting a read of the whole block.

Hashing the whole file after that is wasteful. You need to read (and hash) only as much as needed to demonstrate uniqueness of the file in the set.

The tree concept can be extended to every byte in the file:

https://github.com/kornelski/dupe-krill?tab=readme-ov-file#n...

Re: Hyperspace

#455

Post by pornel »

unclebucknasty wrote: post #271
MBCook wrote: post #180

Earlier quoted context omitted.

You can start with the size, which is probably really unique. That would likely cut down the search space fast. At that point maybe it’s better to just compare byte by byte? You’ll have to read the whole file to generate the hash and if you just compare the bytes there is no chance of hash collision no matter how small. Plus if you find a difference in bytes 1290 you can just stop there instead of reading the whole t…

> which is probably really unique Wonder what the distribution is here, on average? I know certain file types tend to cluster in specific ranges. > maybe it’s better to just compare byte by byte? You’ll have to read the whole file to generate the hash Definitely, for comparing any two files. But, if you're searching for duplicates across the entire disk, then you're theoretically checking each file multiple times, an…

> exceedingly rare

To have a mere one in a billion chance of getting a SHA-256 collision, you'd need to spend 160 million times more energy than the total annual energy production on our planet (and that's assuming our best bitcoin mining efficiency, actual file hashing needs way more energy).

The probability of a collision is so astronomically small, that if your computer ever observed a SHA-256 collision, it would certainly be due to a CPU or RAM failure (bit flips are within range of probabilities that actually happen).

Re: Hyperspace

#456

Post by xenadu02 »

kccqzy wrote: post #438
alwillis wrote: post #401

Just want to mention: Apple ships a modified version of the copy command (good old cp) that supports the ability to use the cloning feature of APFS by using the -c flag.

And in case your cp doesn't support it, you could also do it by invoking Python. Something like `import Foundation; Foundation.NSFileManager.defaultManager().copyItemAtPath_toPath_error_(...)`.

Correct. Foundation's NSFileManager / FileManager will automatically use clone for same-volume copies if the underlying filesystem supports it. This makes all file copies in all apps that use Foundation support cloning even if the app does nothing.

libcopyfile also supports cloning via two flags: COPYFILE_CLONE and COPYFILE_CLONE_FORCE. The former clones if supported (same volume and filesystem supports it) and falls back to actual copy if not. The force variant fails if cloning isn't supported.

Re: Hyperspace

#457

Post by notatallshaw »

karparov wrote: post #447
codemusings wrote: post #408

Earlier quoted context omitted.

John has reiterated multiple times on his podcast that he doesn't want to deal with thousands of support requests when making his apps open source and free. All his apps are personal itches he scratched and he sells them not to make a profit but to make the barrier of entry high enough to make user feedback manageable.

Sorry, that's a BS reason. If you don't want that, just ignore all opened issues. That's it. If you are nice then you put a README that explains this in a sentence or two. If a community forms that wants to fix issues, for example critical ones that could lead to data loss then the community will deal with it, e.g. by forking. Just keeping everything closed is really missing the point of how trust in infra that handl…

As I understand it, from listening to the podcast, a better summary is that if it becomes popular, he wants it to be worthwhile for him to keep working on.

Apps like this can easily bit rot, and more users does often mean more work e.g. answering or filtering emails, finding more edge cases, etc.

From his perspective that means having a income to dedicate time to this. I don't think he's interested in being an "infra" app as you would think of it.

As someone who maintains critical open-source software, I can strongly empathize, even if it’s not an approach I would take.

Re: Hyperspace

#458

Post by notatallshaw »

wubrr wrote: post #446
codemusings wrote: post #408

Earlier quoted context omitted.

John has reiterated multiple times on his podcast that he doesn't want to deal with thousands of support requests when making his apps open source and free. All his apps are personal itches he scratched and he sells them not to make a profit but to make the barrier of entry high enough to make user feedback manageable.

> he doesn't want to deal with thousands of support requests when making his apps open source and free. Who says you have to deal with support requests if you open source something? > All his apps are personal itches he scratched and he sells them not to make a profit but to make the barrier of entry high enough to make user feedback manageable. That makes no sense

> Who says you have to deal with support requests if you open source something?

Almost anyone who has ever maintained popular open-source software, even if dealing with them means putting up a notice that says "Don't ask support questions" and having to delete angrily posted issues.

My understanding from listening to his explanation is he wants to be able to support users and have an income stream to incentivize that.

As an open-source maintainer of a popular piece of software, I'm very empathetic.

Re: Hyperspace

#459

Post by kccqzy »

jlhawn wrote: post #453
kccqzy wrote: post #438

Earlier quoted context omitted.

And in case your cp doesn't support it, you could also do it by invoking Python. Something like `import Foundation; Foundation.NSFileManager.defaultManager().copyItemAtPath_toPath_error_(...)`.

that function name `copyItemAtPath_toPath_error_` sounds like it was ported directly from objective-c!

They might not have ported it. They could have a Python __getattr__ implementation that returns a callable that simply uses objc_msgSend under the hood.

Re: Hyperspace

#460

Post by notatallshaw »

albertzeyer wrote: post #74

I wrote a similar (but simpler) script which would replace a file by a hardlink if it has the same content. My main motivation was for the packages of Python virtual envs, where I often have similar packages installed, and even if versions are different, many files would still match. Some of the packages are quite huge, e.g. Numpy, PyTorch, TensorFlow, etc. I got quite some disk space savings from this. https://githu…

uv does this out of the box, I think other tools (poetry, hatch, pdm, etc.) do as well but I have less experience with the details.
Post reply on HN