The sad history of Unicode printf-style format specifiers in Visual C++ (2019)

Post by bergesenha » Fri, Dec 02, 2022, 7:18 AM UTC

SaddledBounding wrote: post #50

Mikhail_Edoshin wrote: post #30

Earlier quoted context omitted.

This is plain decoding of a trusted string: utf16 decode if (unit = 0xE000) /* one unit */; else /* two units */ utf-8 decode if (unit UTF-16 will take one comparison for all characters up to 0xD7FF; UTF-8 normally takes two. Same when encoding: utf16 encode if (value (We can tune UTF-8 it to take one comparison for ASCII if we expect mostly ASCII.) Things get even more complex when we are to read an untrusted string…

utf-8 has an elegance that utf-16 lacks: utf-8 decode switch (std::countl_one(unit)) { case 0: /* one unit */ break; case 2: /* two units */ break; case 3: /* three units */ break; case 4: /* four units */ break; default: /* not code point boundary */ break; }

Post by manholio » Sat, Dec 03, 2022, 7:15 AM UTC

fomine3 wrote: post #70

manholio wrote: post #8

Earlier quoted context omitted.

What's depressing about the saga of utf8 adoption is that it's such an obvious solution. The ISO2022 standards used the principle since the 70s, and programmers of the era were widely accustomed to extending 8 bit instruction sets to multibyte instructions. It's one of those cases where you need to hit your head hard and repeatedly until you bite the bullet and accept that the cost of multibyte (for example, strlen()…

ISO 2022 isn't beautiful solution like UTF-8 self sync encoding. Backward compatibility is also great (so called UTF FSS). It was designed by smart Plan9 people.