APFS does not normalize Unicode filenames
1–10 of 148 posts
Re: APFS does not normalize Unicode filenames
#2This is a big change. I guess they now decided that compatibility with external systems is a more important goal than end-user-friendliness.
It’s a reasonable decision to come to (especially for iOS where end-users don’t ever really interact with the filesystem directly), but it will cause quite a bit of churn in the short term.
Re: APFS does not normalize Unicode filenames
#3I agree that this is a good change. Unicode, normalisation, character encodings, etc. should really be handled at the presentation layer, and everything below that just treats filenames as sequences of bytes, perhaps with one or two exceptions like '/' and \0.
It is interesting to consider a theoretical system in which paths are represented in 0-terminated count-length format (e.g. "foo/bar/baz/myfile.txt" would be "\003foo\003bar\003baz\012myfile.txt\000"), truly allowing any byte in a filesystem node's name, although that might be going a little bit too far.
Things are much easier for the file system if it can just treat names as bags of bytes.
If you're really talking about bags (unordered sets), that would certainly make for an interesting filesystem since filename.txt, filemane.txt, and maletent.fix would all be the same...
Re: APFS does not normalize Unicode filenames
#4This seems especially bad because US-based developers who don't test with unicode filenames might not come across this issue, leaving all their non-English-speaking customers broken. (Not that this excuses such developers in any way.)
It also means that, yet again, every app will need to be updated for a new version of iOS. Makes me wonder how many apps will be left behind if not updated? Thousands? Hundreds of thousands? Millions?
Re: APFS does not normalize Unicode filenames
#5This is a big change. I guess they now decided that compatibility with external systems is a more important goal than end-user-friendliness. It’s a reasonable decision to come to (especially for iOS where end-users don’t ever really interact with the filesystem directly), but it will cause quite a bit of churn in the short term.
[deleted]
Re: APFS does not normalize Unicode filenames
#6[deleted]
Re: APFS does not normalize Unicode filenames
#7I agree that this is a good change. Unicode, normalisation, character encodings, etc. should really be handled at the presentation layer, and everything below that just treats filenames as sequences of bytes, perhaps with one or two exceptions like '/' and \0. It is interesting to consider a theoretical system in which paths are represented in 0-terminated count-length format (e.g. "foo/bar/baz/myfile.txt" would be "…
Incidentally, your theoretical system is exactly how the DNS stores domain names.
Re: APFS does not normalize Unicode filenames
#8This seems especially bad because US-based developers who don't test with unicode filenames might not come across this issue, leaving all their non-English-speaking customers broken. (Not that this excuses such developers in any way.) It also means that, yet again, every app will need to be updated for a new version of iOS. Makes me wonder how many apps will be left behind if not updated? Thousands? Hundreds of thous…
Except filesystem experts like Dropbox, developers probably shouldn't be letting users name their files.
Re: APFS does not normalize Unicode filenames
#9Seems to me that Apple would be smart to hook all of the file functions and survey and/or alert on this situation somehow.
I only just learnt about this unicode normalisation recently looking at ZFS which has options for it I had never seen until reading the Ubuntu Root FS on ZFS guides which talk about setting it.
Re: APFS does not normalize Unicode filenames
#10This seems especially bad because US-based developers who don't test with unicode filenames might not come across this issue, leaving all their non-English-speaking customers broken. (Not that this excuses such developers in any way.) It also means that, yet again, every app will need to be updated for a new version of iOS. Makes me wonder how many apps will be left behind if not updated? Thousands? Hundreds of thous…
You should not be using anything other than UUIDs or integers for file names. Maintain your own mapping in a database or file.
Using a network value or a value returned by an API is just asking for trouble.
If a user names a file that will be hidden behind a URL the same advice applies. If not then the user can use any sequence of bytes they want and you shouldn't care.