11–20 of 73 posts
Earlier quoted context omitted.
In my experience, most technical people these days still don't know the difference between UTF-8 and Unicode for example.
Is this the best reference? Too detailed? https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
Earlier quoted context omitted.
Maybe they didn't want to be early adopters of what could turn out to be a second fad like UCS-2 was the first time.
What's depressing about the saga of utf8 adoption is that it's such an obvious solution. The ISO2022 standards used the principle since the 70s, and programmers of the era were widely accustomed to extending 8 bit instruction sets to multibyte instructions. It's one of those cases where you need to hit your head hard and repeatedly until you bite the bullet and accept that the cost of multibyte (for example, strlen()…
Countries like Japan had already solved encoding differently while US was still using ASCII. (See JIS encoding).
As time passed tho, the byte savings matter less. Also combined with transfer compression makes the encoding size differences negligible.
I am kinda surprised how much Windows likes its wchar's. Windows had support for multi-byte codepages since early days, and it has ways to indicate compatibility level for individual apps. It seems the solution would be simple: once UTF-8 introduced (back in 1996!), add new "UTF-8" codepage, add new "SetDefaultCodepageToUnicode()" function, and tell everyone that wchar_t is legacy and should not be used anymore. They…
I have asked myself the same question. However, I observed that the switch from UCS-2 to true UTF-16 was already very bumpy, with strange bugs and tons of edge cases that only got significantly better in Windows 8. A forced switch to UTF-8 would likely have broken ANSI strings. Languages such as PHP didn't get true UTF-8 support until 2015, Python in 2008 (and that switch took a decade), and Java and JavaScript conti…
I am kinda surprised how much Windows likes its wchar's. Windows had support for multi-byte codepages since early days, and it has ways to indicate compatibility level for individual apps. It seems the solution would be simple: once UTF-8 introduced (back in 1996!), add new "UTF-8" codepage, add new "SetDefaultCodepageToUnicode()" function, and tell everyone that wchar_t is legacy and should not be used anymore. They…
I am kinda surprised how much Windows likes its wchar's. Windows had support for multi-byte codepages since early days, and it has ways to indicate compatibility level for individual apps. It seems the solution would be simple: once UTF-8 introduced (back in 1996!), add new "UTF-8" codepage, add new "SetDefaultCodepageToUnicode()" function, and tell everyone that wchar_t is legacy and should not be used anymore. They…
UTF-16 is simpler and more efficient to handle than UTF-8.
Only if you never go out of the basic multilingual plane. And then find out so many bugs when people start using emojis.
I am kinda surprised how much Windows likes its wchar's. Windows had support for multi-byte codepages since early days, and it has ways to indicate compatibility level for individual apps. It seems the solution would be simple: once UTF-8 introduced (back in 1996!), add new "UTF-8" codepage, add new "SetDefaultCodepageToUnicode()" function, and tell everyone that wchar_t is legacy and should not be used anymore. They…
I think there was also some compatibility concerns, as Windows for a while only had a single global "ASCII" character set used by all apps[2], so you'd have issues with actual legacy apps expecting a real legacy code page breaking / having scrambled text if you tried to use UTF8 as the "legacy" encoding. Or the fun idea of an app that is clever enough to know that multibyte exists, but dumb enough to know they can only be one or two bytes long, and allocating buffers accordingly. I think there might have been some issues with how things like the clipboard handled text as well that assumed all "ASCII" apps used the same encoding. (Disclaimed: old memories of reading blogs, may be bollocks). At some point things must have been fixed so multiple character sets could be used though.
[1] But not on 9x, which was "ASCII" only, at least until late in the day when a Unicode compatibility library was created.
[2] The "Language for non-Unicode programs" as the Control Panel calls it.
I am kinda surprised how much Windows likes its wchar's. Windows had support for multi-byte codepages since early days, and it has ways to indicate compatibility level for individual apps. It seems the solution would be simple: once UTF-8 introduced (back in 1996!), add new "UTF-8" codepage, add new "SetDefaultCodepageToUnicode()" function, and tell everyone that wchar_t is legacy and should not be used anymore. They…
UTF-16 is simpler and more efficient to handle than UTF-8.
The historical reasons of why UTF-16 exists are fully understandable, but by now it is clear that it's the worst of both worlds.
Earlier quoted context omitted.
I have asked myself the same question. However, I observed that the switch from UCS-2 to true UTF-16 was already very bumpy, with strange bugs and tons of edge cases that only got significantly better in Windows 8. A forced switch to UTF-8 would likely have broken ANSI strings. Languages such as PHP didn't get true UTF-8 support until 2015, Python in 2008 (and that switch took a decade), and Java and JavaScript conti…
How many Windows users do benefit significantly from using UCS-2 encoding, I wonder? Chinese and Hindi (along with Japanese, South Korean and various Arabic language) speakers would surely come close to outnumbering those of us using European alphabets (I gather that Cyrillic & Greek have single-byte encodings, but maybe they mostly use UCS-2 these days too?). On that basis it seems a reasonable choice as the default…
I have no problem with saving text files in UTF-8 and normally do. I just wish SCSU was handled by COM by default as well and not only sometimes.
I remember this hoopla. And windows wasn't the only victim of using "two bytes for a char" for unicode support. Java fell in this trap too. The team was split in deciding "whether to waste a single byte per character for string" -- that's in their words, and then decided to rather be ready for the future. That's in 1995. And then everyone later watched UTF-8 take over the world. The PHP team also wasted their version…
Earlier quoted context omitted.
UTF-16 is simpler and more efficient to handle than UTF-8.
I feel like you're confusing it with UTF-32, which is fixed-length. UTF-16 is neither fixed-length, like UTF-32, nor compatible with the old ASCII encoding, like UTF-8. The historical reasons of why UTF-16 exists are fully understandable, but by now it is clear that it's the worst of both worlds.