Live data from Hacker News

APFS is not safe to use with names which have Unicode normalisation issues

eclecticlight.co

31–40 of 191 posts

Re: APFS is not safe to use with names which have Unicode normalisation issues

#31

Earlier quoted context omitted.

"without warning"? Not really. This has been discussed to death on ATP last year. Also, in regards to the headline, there are tens or maybe over a hundred million non-english speakers using iOS already running APFS...

iOS never exposes low-level filenames to the user.

One of the new features in iOS 11 presented yesterday is the File App. It provides access to all files (at some level) in the system and on networked systems including iCloud, Dropbox etc.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#32

Earlier quoted context omitted.

Blame the encodings that came before it: Latin-1 and Windows-1252, and all the other ones. Hindsight is always 20/20, y'know.

I'm just being sarcastic: should we also blame the computers of old for not being able to handle more than 255 characters ?

Not really. In a barely-networked and severely resource-constrained world (kilobytes of memory, not gigabytes), this decision made sense. Shift happens.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#33
post #4

The title is click-bait and over-dramatises the issue. The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here). If you want to see the kind of issues that path normalisation brings, check out this: https://github.com/thibaudgg/rb-fsevent/blob/master/ext/fsev... I'd like to believe that most developer would prefer the current behaviour over t…

> The title is click-bait and over-dramatises the issue. Sadly it's not. > The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here). That is incorrec. APFS treats filenames as utf-8 strings and depending on the API you are using normalization is still taking place but on different levels. For instance all Cocoa APIs will perform normalization…

So it is correct then, the file system doesn't concern itself with normalization. File names are stored internally as utf-8 strings which are just a sequence of bytes.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#34

Earlier quoted context omitted.

iOS never exposes low-level filenames to the user.

One of the new features in iOS 11 presented yesterday is the File App. It provides access to all files (at some level) in the system and on networked systems including iCloud, Dropbox etc.

That only ever surfaces filenames through cocoa APIs that underwent normalization.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#35
post #33

Earlier quoted context omitted.

> The title is click-bait and over-dramatises the issue. Sadly it's not. > The choice of APFS is that a filename is a sequence of bytes. Nothing more, nothing less (feel free to correct me if I'm wrong here). That is incorrec. APFS treats filenames as utf-8 strings and depending on the API you are using normalization is still taking place but on different levels. For instance all Cocoa APIs will perform normalization…

So it is correct then, the file system doesn't concern itself with normalization. File names are stored internally as utf-8 strings which are just a sequence of bytes.

By that logic that was true for HFS+ as well.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#37
post #20

Earlier quoted context omitted.

I don't remember those having multiple representations for the same character in the same encoding or normal forms...

Indeed not: there is an almost infinite number of things that Latin-1 did not have; e.g. any characters beyond Western Europe. While keeping the 128-256 block compatible was a part of early Unicode (hence the "one-glyph" é ), having composed characters was a Unicode primary design goal (hence e and the composing accent). A pure Unicode implementation would have been better, maybe; what we have instead is one that has…

Well, composed characters may or may not have been a good idea, although they're a pain for text search if you ask me.

But why oh why did they have to specify FOUR normalization forms?

Ref: http://www.unicode.org/reports/tr15/#Norm_Forms

Re: APFS is not safe to use with names which have Unicode normalisation issues

#38
post #37

Earlier quoted context omitted.

Indeed not: there is an almost infinite number of things that Latin-1 did not have; e.g. any characters beyond Western Europe. While keeping the 128-256 block compatible was a part of early Unicode (hence the "one-glyph" é ), having composed characters was a Unicode primary design goal (hence e and the composing accent). A pure Unicode implementation would have been better, maybe; what we have instead is one that has…

Well, composed characters may or may not have been a good idea, although they're a pain for text search if you ask me. But why oh why did they have to specify FOUR normalization forms? Ref: http://www.unicode.org/reports/tr15/#Norm_Forms

¯\_(ツ)_/¯

Re: APFS is not safe to use with names which have Unicode normalisation issues

#39
post #33

Earlier quoted context omitted.

So it is correct then, the file system doesn't concern itself with normalization. File names are stored internally as utf-8 strings which are just a sequence of bytes.

By that logic that was true for HFS+ as well.

HFS+ did perform normalization on a file system level and stored strings as utf-16 so it's not true for HFS+.

Re: APFS is not safe to use with names which have Unicode normalisation issues

#40
So, can somebody answer this, is this right on APFS or not?

    rsync -rltv --iconv=utf-8-mac,utf-8 from_a to_b
and similar sshfs: https://github.com/osxfuse/sshfs/issues/14#issuecomment-1859...

I need to use that iconv options everytime when dealing with umlauts and maclinux communication. So do I still need that on APFS or not?

Post reply on HN