APFS is not safe to use with names which have Unicode normalisation issues
1–10 of 191 posts
Re: APFS is not safe to use with names which have Unicode normalisation issues
#2Would be interesting to know why Apple made the decision not to normalize on a file system level then. Just an argument based on separation of concerns?
Re: APFS is not safe to use with names which have Unicode normalisation issues
#3Would be interesting to know why Apple made the decision not to normalize on a file system level then. Just an argument based on separation of concerns?
Normalizing on the file system level in the past was a mistake. Normalizing Unicode is not an easy task, it belongs into userspace and not into a kernel driver.
Other systems consider filename a string of octets and interpretation what these octets mean is left for userspace to decide.
For Linus Torvalds colorful opinions on HFS+, see here: https://plus.google.com/+JunioCHamano/posts/1Bpaj3e3Rru (in the comments).
Re: APFS is not safe to use with names which have Unicode normalisation issues
#4The title is click-bait and over-dramatises the issue.
The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here).
If you want to see the kind of issues that path normalisation brings, check out this: https://github.com/thibaudgg/rb-fsevent/blob/master/ext/fsev...
I'd like to believe that most developer would prefer the current behaviour over that proposed by the article (normalisation).
If there are issues in Apple's Finder or other high level APIs, I'd like to believe they will be fixed before the final release. These are fixable bugs, IMHO.
Re: APFS is not safe to use with names which have Unicode normalisation issues
#5The title is click-bait and over-dramatises the issue. The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here). If you want to see the kind of issues that path normalisation brings, check out this: https://github.com/thibaudgg/rb-fsevent/blob/master/ext/fsev... I'd like to believe that most developer would prefer the current behaviour over t…
The problem seems to come from the high-level APIs doing normalization. So if you have two files in a directory, one normalized and one not-normalized and open the non-normalized file, the high-level API will then normalize that filename and you open the wrong file.
I've always disliked the practice of messing around with file paths (storing them, concatenating them, etc). I preferred the way that the Classic MacOS typically dealt with filesystem references and aliases instead of file paths. This also meant users can rename or move files and references still work.
Re: APFS is not safe to use with names which have Unicode normalisation issues
#6I'm happy that the filesystem treats name as sequences of bytes. Normalization should happen at a higher level.
Re: APFS is not safe to use with names which have Unicode normalisation issues
#7To be honest, I blame Unicode. Why allow different representations for the same character, and then provide a normalized form anyway, except it's not one normalized form but several? Sounds like job security to me.
Re: APFS is not safe to use with names which have Unicode normalisation issues
#8The title is click-bait and over-dramatises the issue. The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here). If you want to see the kind of issues that path normalisation brings, check out this: https://github.com/thibaudgg/rb-fsevent/blob/master/ext/fsev... I'd like to believe that most developer would prefer the current behaviour over t…
The problem seems to come from the high-level APIs doing normalization. So if you have two files in a directory, one normalized and one not-normalized and open the non-normalized file, the high-level API will then normalize that filename and you open the wrong file. I've always disliked the practice of messing around with file paths (storing them, concatenating them, etc). I preferred the way that the Classic MacOS t…
[deleted]
Re: APFS is not safe to use with names which have Unicode normalisation issues
#9The real problem isn't whether filename normalization is a good/bad thing, the problem is Apple used to do it one way and is now switching without warning to doing it the other way.
It's the logical end product of the odyssey from Apple's original philosophy of a resource and data fork model for files to the UNIX stream-of-bytes model for files. The UNIX model traditionally kept metadata about files separate (anyone else remember naughtily hand-editing directory files with sed(1), back in the day?) but gradually picked up a bunch of new stuff that has to be stored somewhere. Meanwhile, the Mac platform adopted UNIX binaries with the move to NeXTStep underpinnings (we're going back to the late 90s here, and the adoption of OSX over traditional MacOS), obviating the need for the original resource bundle, which got rid of those annoying errors about missing bundle bits but left us with a legacy of .DS_Store turdlets in every directory to hold file metadata that was formerly stored in the resource fork.
As a destination it's laudable — design consistency is almost always laudable — but it's the end goal of a very messy process and it looks like APFS isn't quite there yet; NSFileManager and NSURL need some way of distinguishing files with different unicode representations, and the Finder in particular needs to be robust. I'm guessing this is going to be fixed when High Sierra finally ships, but isn't supported in Sierra at present, hence the OP's alarm at the situation.
Re: APFS is not safe to use with names which have Unicode normalisation issues
#10There's a potentially useful discussion to be had on normalization but the title is pure clickbait hyperbole. HFS+ is the only filesystem in common use which performs Unicode normalization and a statement that bold would require at least some evidence that Windows, Linux, etc. are only usable by English speakers.
My position on this is mixed. I've had to write code to deal with normalization changes in archives and it's quite tedious. The HFS+ approach of normalization is in many ways the best choice considered in isolation but in practice it's really expensive to support since most filesystems, tools, APIs, etc. predate Unicode and everyone else chose the bag of bytes approach.