111–120 of 170 posts
Given that the article says that everyone should use utf8mb4, then why don't/haven't they just fixed it? Should be easy enough, and probably safe, too.
They have. It's the default in MySQL 8.
> If you need a database, don’t use MySQL or MariaDB. Use PostgreSQL. This is stupid. There are pros and cons to every database. For example, MySQL allows you to set a trigger on an op, PostgreSQL requires you to write a function first.
Talk about stupid, run this query on both Mysql and Postgres:
select 0 = 'banana';
Make your own conclusion.
Earlier quoted context omitted.
With utf-8 you give up any hope of O(1) string operations. With utf-16 you can get O(1) string operations at a cost of doing them on code points rather than graphemes. If greater than 95% of the strings your language/program will ever see are going to be a single code point that trade off seems worth it. I think a lot of people totally discount the enormous cost that Latin-1 users are paying for CJK (etc) support the…
Except that with utf-16 you still have to give up any hope of O(1) string operations, unless you don't care about accuracy. Only UTF-32 allows O(1) index based operations. For example, 🧐 (aka face with monocle) needs 2 UTF-16 'characters', since the UTF code point is U+1F9D0, or, in UTF-8: F0 9F A7 90. And this ignored the point that others have made about combining characters, such as woman farmer Unicode: U+1F46…
A woman farmer requires the "woman" and "ear of rice" emoji, with a zero-width joiner character between them. To change the skin tone from the default yellow, a "medium skin tone" (type 4) modifier is added after the woman, but before the joiner.
So the sequence "U+1F469 U+1F3FD U+200D U+1F33E" represents "woman skin-tone-4 joiner ear-of-rice". And in the UTF-8 bytes, the first four bytes are "woman", the next four are "skin-tone-4", then three for "zero-width-joiner", and finally four for "ear-of-rice".
Emojipedia helpfully lists codepoints for each multi-character emoji: https://emojipedia.org/female-farmer/
I use utf8 for "normal" field like name, address, etc.. (don't want smiley as first name). And use utf8mb4 for comment for example. But I need filter/validate each inputs (and refuse when there is 4 bytes characters).
Earlier quoted context omitted.
They'd kind of what happened in python 3.3. The internal representation is flexible, depending on the string contents. It may be just 1byte ASCII, or it may be utf32 if needed: https://www.python.org/dev/peps/pep-0393/
Java 11 is going to a similar model.
Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane, and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support, and enough people care about them that adoption of this basic level of Unicode is driven, and so non-English speakers are inadverten…
Earlier quoted context omitted.
They have. It's the default in MySQL 8.
Sorry, should have bern more clear: why didn't they fix the bug in utf8 so that it is correct (and would do what utf8mb4 does now)? It's just that the obvious choice of 'utf8' has been plain wrong for a long time.
>Once MySQL published this invalid character set, it could never fix it: that would force every user to rebuild every database.
I use utf8 for "normal" field like name, address, etc.. (don't want smiley as first name). And use utf8mb4 for comment for example. But I need filter/validate each inputs (and refuse when there is 4 bytes characters).