Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
1–10 of 65 posts
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#2Or otherwise said: Surrogate Pairs are used in UTF-16 (which uses two bytes per character, so it can encode up to 65536 characters) to encode Unicode characters that have code points that can't be encoded using just two bytes.
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#3I think it's hilarious that the event viewer XML gets borked.
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#4I remember that in Mac OS X times, sometime between OS X v10.1 and 10.4, a system upgrade caused a bunch of unicode named files to become inaccessible/untouchable (but still present with a directory listing). At the time I didn't have the skills to figure out what had happened. I'm still curious to know if it was an intended breaking change.
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#5Aha! Found the name of my next album. Try downloading me on Napster now!
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#6I think it's hilarious that the event viewer XML gets borked.
I am not 100% sure but mmc.exe has not been updated for years and it must be relying on WebBrowser control of Internet Explorer. Yes, IE is still alive in Windows.
https://learn.microsoft.com/en-us/previous-versions/windows/...
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#7Or otherwise said: Surrogate Pairs are used in UTF-16 (which uses two bytes per character, so it can encode up to 65536 characters) to encode Unicode characters that have code points that can't be encoded using just two bytes.
Yep. The quirk here is that the surrogates, that are merely enablers for other characters, can be paired with each other. With the absence of other valid characters, they are not enabling anything. One assumes there is a validation but it does not exist here.
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#8Yet another reason to use Linux everywhere. It is 2025 and Windows (and probably Mac) users have to deal with weird Unicode filesystem issues. Good luck puting Chinese characters or emoticons into filenames.
Ext4 filename has maximal length 255 characters. That is the only legacy limit you have to deal with as a Linux user. And even that can be avoided by using more modern filesystems.
And we get filesystem level snapshots etc...
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#9Or otherwise said: Surrogate Pairs are used in UTF-16 (which uses two bytes per character, so it can encode up to 65536 characters) to encode Unicode characters that have code points that can't be encoded using just two bytes.
Yep. The quirk here is that the surrogates, that are merely enablers for other characters, can be paired with each other. With the absence of other valid characters, they are not enabling anything. One assumes there is a validation but it does not exist here.
There is no validation on the file system level because file names in NTFS are sequences of arbitrary 16-bit values, similar to how on Unix file systems, file names are sequences of arbitrary 8-bit values. Arguably the situation on Unix is worse, because there the interpretation and validity depends on the current locale.
Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read
#10Yet another reason to use Linux everywhere. It is 2025 and Windows (and probably Mac) users have to deal with weird Unicode filesystem issues. Good luck puting Chinese characters or emoticons into filenames. Ext4 filename has maximal length 255 characters. That is the only legacy limit you have to deal with as a Linux user. And even that can be avoided by using more modern filesystems. And we get filesystem level sna…
You have the same, if not worse, issue on Linux with filenames that aren’t valid UTF-8 sequences. Not to mention that on Linux switching the locale may change the interpretation of filenames as characters, which isn’t the case with NTFS.