Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MacOS may lose data on APFS-formatted disk images

bombich.com

Post reply on HN
Print view View original

101–110 of 193 posts

Re: MacOS may lose data on APFS-formatted disk images

#101

Post by loeg »

toasterlovin wrote: post #16
torstenvl wrote: post #9

Yes, fairly limited impact, but still. Data loss is fucking important. We aren't talking about an occasional stutter when moving windows or a sound driver that sometimes needs a reboot to resume working. This is a major bug. I would really really like it if Apple would get their shit together so that people who actually rely on their computers to work correctly can upgrade at some point.

As a counterpoint, APFS was deployed to almost a billion devices over the span of several months when it was release and, IIRC, this is the first major issue.

Yeah. Frankly, the APFS rollout was incredibly smooth for a new filesystem.

Re: MacOS may lose data on APFS-formatted disk images

#102

Post by praseodym »

zbentley wrote: post #88
rlkf wrote: post #15

Is the incident with the matching checksum that he mentions because APFS only checksum metadata (which are on preallocated space on the image), or is it his own checksum (say sha1sum), I wonder? It seems strange that the filesystem driver would cache 500 GB of sequentially written data in RAM.

> It seems strange that the filesystem driver would cache 500 GB of sequentially written data in RAM. That was the most interesting/worrying part of TFA, and I would love to see how the checksum tests were conducted clarified in the text. Presumably, the "md5" commandline tool has no special fallback to the filesystem checksum cache (if it does, rather a lot of my life has been a lie, I'm afraid). Since that's the ca…

The video embedded in TFA shows md5 reporting identical checksums before unmounting the disk image, so it must be reading the data from a cache.

Re: MacOS may lose data on APFS-formatted disk images

#103

Post by JustSomeNobody »

alwillis wrote: post #54
kbumsik wrote: post #40

Two bugs are described in this article: 1. An APFS volume's free space doesn't reflect a smaller amount of free space on the underlying disk 2. The diskimages-helper application doesn't report errors when write requests fail to grow the disk image These are not even complex problems of the new format. It is just Apple forgot to have basic checks. It is like the root access with an empty password incident happened 2 m…

This is by far the best explanation of what’s happening at Apple and why we’re noticing these bugs and problems: https://www.imore.com/understanding-apple-scale

Rene Ritchie is very biased. He has to be as his livelihood depends on Apple. I take his words with many grains of salt. The only reason he's quoting Sinofsky is because it gives him a way of excusing Apple's software stumbles of late.

Re: MacOS may lose data on APFS-formatted disk images

#104

Post by dunham »

eropple wrote: post #57
blattimwind wrote: post #51

Earlier quoted context omitted.

TBH an I/O system where having a file system inside a loopback device on another file system is faster than using said file system directly in the first place sounds kinda broken-ish/poorly scaling to me.

Having Spotlight ignore .git directories and the like is probably wise, I would agree with that. But it's text, even if it's basically garbage text (from a user perspective). So I can understand how a sparsebundle is a decent end-around. The Finder in general ends up basically being useless for me for similar reasons; I have dozens of random dependency files I don't even recognize pop up in "All My Files".

Spotlight ignores hidden directories (e.g. .git) and directories whose names end in .noindex. You can create a file in one with a unique name and try to mdfind it to verify this.

Re: MacOS may lose data on APFS-formatted disk images

#105

Post by jchb »

praseodym wrote: post #95

APFS not having block or file-level checksums really seems like a big oversight to me. While the filesystem’s designers considered the hardware-level guarantees to be sufficient [1], this issue shows that there is an entire class of problems that they have not considered. Disk images and loopback-mounted filesystems or even disk-level cloning introduce additional layers of complexity where a filesystem can be silentl…

Apple HFS+ didn't have file data checksums either. The default filesystem on most Linux distributions, ext4 doesn't, it just stores checksums of the file metadata, not the file data. Same story with Windows NTFS filesystem. Microsofts newer ReFS filesystem has file data checksums disabled by default. So it seems like a tradeoff that most of the major operating system are making. Most likely related to performance.

Edit: macOS disk images do have a checksum of the whole image data though. The issue mentioned in the article seems to be caused by an oversight in the disk image helper app, rather than in the APFS filesystem itself.

Re: MacOS may lose data on APFS-formatted disk images

#106

Post by eropple »

dunham wrote: post #104
eropple wrote: post #57

Earlier quoted context omitted.

Having Spotlight ignore .git directories and the like is probably wise, I would agree with that. But it's text, even if it's basically garbage text (from a user perspective). So I can understand how a sparsebundle is a decent end-around. The Finder in general ends up basically being useless for me for similar reasons; I have dozens of random dependency files I don't even recognize pop up in "All My Files".

Spotlight ignores hidden directories (e.g. .git) and directories whose names end in .noindex. You can create a file in one with a unique name and try to mdfind it to verify this.

Does it ignore them, or does it traverse them and throw them out?

Anecdotally it certainly seems like indexing is slower on my dev drive than anywhere else, so I'm curious.

Re: MacOS may lose data on APFS-formatted disk images

#107

Post by hueving »

zbentley wrote: post #85
jandrese wrote: post #80

Earlier quoted context omitted.

My wife's laptop suddenly decided that the boot drive was corrupt a few weeks after she updated to APFS. None of the recovery tools were of any use. We had to reinstall the OS and pull the files from a backup. This story did not make it to Hacker News.

Anecdata != filesystem issue. If you have more information about the problem you encountered and how it implicates/interacts with APFS, please do link to it. Otherwise, bug reports via circumstantial evidence are, while not inherently false, certainly suspect.

You missed the point. The anecdote was to illustrate how even power users might be working around filesystem bugs so a lack of bug reports specifically mentioning APFS is certainly not proof that there aren't problems.

Re: MacOS may lose data on APFS-formatted disk images

#108

Post by gigatexal »

alwillis wrote: post #56
gigatexal wrote: post #31

Should have paid Oracle and just put ZFS on OS X.

Apple killed ZFS on macOS nearly 10 years ago. It wouldn’t make sense on a watch or a phone anyway: http://www.zdnet.com/article/mac-zfs-is-dead-rip/

200B in the bank and they couldn’t find a way to port it to other platforms? I think they could.

Re: MacOS may lose data on APFS-formatted disk images

#109

Post by yborg »

voldemort1968 wrote: post #34

Not to downplay the importance of this, but it reads as clickbait that you wait until the second paragraph to say "oh yeah, it's only sparsebundles. Just those things that almost nobody uses."

Yes, nobody - like for example Apple, to implement Time Machine.

But now perhaps I better understand why Time Machine backups aren't supported on APFS.

Re: MacOS may lose data on APFS-formatted disk images

#110

Post by ghostcluster »

toasterlovin wrote: post #16
torstenvl wrote: post #9

Yes, fairly limited impact, but still. Data loss is fucking important. We aren't talking about an occasional stutter when moving windows or a sound driver that sometimes needs a reboot to resume working. This is a major bug. I would really really like it if Apple would get their shit together so that people who actually rely on their computers to work correctly can upgrade at some point.

As a counterpoint, APFS was deployed to almost a billion devices over the span of several months when it was release and, IIRC, this is the first major issue.

It irrecoverably destroyed my partition on upgrade to High Sierra.
Post reply on HN