Live data from Hacker News

Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

zaferbalkan.com

61–65 of 65 posts

Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

#61
post #50
post #22

Earlier quoted context omitted.

Oh, great. Can you also share the locale? I'll write another Postscriptum section then.

LANG=en_IE.UTF-8 LANGUAGE=en_IE:en LC_CTYPE="en_IE.UTF-8" LC_NUMERIC="en_IE.UTF-8" LC_TIME="en_IE.UTF-8" LC_COLLATE="en_IE.UTF-8" LC_MONETARY="en_IE.UTF-8" LC_MESSAGES="en_IE.UTF-8" LC_PAPER="en_IE.UTF-8" LC_NAME="en_IE.UTF-8" LC_ADDRESS="en_IE.UTF-8" LC_TELEPHONE="en_IE.UTF-8" LC_MEASUREMENT="en_IE.UTF-8" LC_IDENTIFICATION="en_IE.UTF-8" LC_ALL=

Not the OP, but have you tried LANG=C and getting rid of the rest?

Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

#62
post #28

Earlier quoted context omitted.

Microsoft never implements a standard, they only ever implement their own shit. Sometimes it's a close enough parody of a standard to fool superficial onlookers, but that's as close as you'll ever get.

Java, NT, .NET, "wide" C and C++ and a few others from the same time frame ended up with WTF-16 because surrogate pairs didn't exist when they were designed. They were designed with UCS-2, which is a fixed-length encoding. Unicode 2.0 then extended that to be variable length (16/32-bit) using surrogate pairs and that's where all the systems come from which don't validate surrogate pairs.

Is that still true with dotnet? Especially since it has a surrogate pair api and a rune api? https://learn.microsoft.com/de-de/dotnet/api/system.char.iss... https://learn.microsoft.com/en-us/dotnet/api/system.text.run...

Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

#63
post #28

Earlier quoted context omitted.

Microsoft never implements a standard, they only ever implement their own shit. Sometimes it's a close enough parody of a standard to fool superficial onlookers, but that's as close as you'll ever get.

Java, NT, .NET, "wide" C and C++ and a few others from the same time frame ended up with WTF-16 because surrogate pairs didn't exist when they were designed. They were designed with UCS-2, which is a fixed-length encoding. Unicode 2.0 then extended that to be variable length (16/32-bit) using surrogate pairs and that's where all the systems come from which don't validate surrogate pairs.

Surrogate pairs existed then, they just ignored them

Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

#64
post #43

Earlier quoted context omitted.

Where did they think Cuneiform was going to fit? Even with unihan the BMP was getting consumed fast.

In the beginning Unicode was only meant to be used for extant languages afaik.

Yes, the clusterfuck that is CJK unification would not have been done if it was known that Unicode was going to go beyond 16-bit anyway.

Re: Understanding Surrogate Pairs: Why Some Windows Filenames Can't Be Read

#65
post #28

Earlier quoted context omitted.

Microsoft never implements a standard, they only ever implement their own shit. Sometimes it's a close enough parody of a standard to fool superficial onlookers, but that's as close as you'll ever get.

>Microsoft never implements a standard Win32 ?

They don't even implement that "standard" correctly according to their own documentation.
Post reply on HN