Counterpoint: emojis are bad and should be discouraged
Why?
In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
61–70 of 170 posts
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#62Earlier quoted context omitted.
ok - but parent referred to "utf8 character set". As if things weren't confusing enough already!
Can't reply to the sibling comment by Chris, but there isn't a "utf8 character set". There's just different encodings of a character set. Even after hunting around after going to the 404 you pointed me to, I still get this: utf8mb4: A UTF-8 encoding of the Unicode character set using one to four bytes per character. utf8mb3: A UTF-8 encoding of the Unicode character set using one to three bytes per character. So I do…
Not sure why the reply link is sometimes missing, but in my experience if you open just that comment (via the “N minutes ago” link) the textbox for replying is there and works. Perhaps the link is omitted to discourage rapid conversation?
I fixed the link almost immediately—pesky inclusion of the dot in “.)”
---
The SQL specification (I cite SQL-92) speaks of character sets (and uses the keyword CHARACTER SET). That outside of SQL, Unicode is the character set which happens to have multiple encodings, is immaterial; in SQL, each Unicode encoding is a CHARACTER SET—`CHARACTER SET utf8`, `CHARACTER SET utf8mb4`, whatever. This is the sense in which I used the term “character set”, the SQL sense.
Before UCS-2 was proved insufficient, that encoding was actually called UNICODE, Wikipedia tells me.
Concerning MySQL, utf8mb3 is an alias for utf8 which was introduced after utf8. I’m guessing it was introduced simultaneously with utf8mb4.
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#63It's a code smell when what is obviously and intuitively stated, is in fact, incorrect.
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#64Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane, and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support, and enough people care about them that adoption of this basic level of Unicode is driven, and so non-English speakers are inadverten…
> Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane
When can we have user-defined glyphs/emoji, where SVG is embedded in the unicode string?
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#65Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane, and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support, and enough people care about them that adoption of this basic level of Unicode is driven, and so non-English speakers are inadverten…
> and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support
If I had to guess, they did it for space and performance reasons so that their benchmark results are a tiny bit better than other RDBMs. They must've figured that their limited utf8 encoding was enough. And they reasoned the immediate benefits on benchmark tests outweighed future considerations.
I can't imagine any other reason for not implementing the utf-8 encoding completely and covering the unicode character set in its entirety.
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#66Earlier quoted context omitted.
I am pretty sure it was not because of any optimization but that the real reason was to make sure a VARCHAR(255) would still fit within the maximum field size of InnoDB (768 bytes) at the time. Since then MySQL has added ROW_FORMAT=DYNAMIC to InnoDB which allows for larger values, but that probably required major changes to InnoDB.
There is usually a good reason for doing these kind of things. But then don't call it UTF8 encoding ! It doesn't do that. It stores the most popular characters out of a larger address space. Call it UTF8-MB3 maybe from the start and let the user decide if having varchar(255) is worth the tradeoff. Thats the trouble with MySQL: doing silently the wrong thing, and thus violating the principle of least surprise. There i…
In the case of MySQL (especially MySQL in 2003) bad design and silly behavior isn’t much of a surprise though :)
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#67Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane, and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support, and enough people care about them that adoption of this basic level of Unicode is driven, and so non-English speakers are inadverten…
> Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane When can we have user-defined glyphs/emoji, where SVG is embedded in the unicode string?
You’d just define two code points that switches into and out of SVG mode. (The second is only necessary for performance, so you can scan quickly without needing any sort of XML parser!)
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#68> If you need a database, don’t use MySQL or MariaDB. Use PostgreSQL. This is stupid. There are pros and cons to every database. For example, MySQL allows you to set a trigger on an op, PostgreSQL requires you to write a function first.
I agree that that point sounds too rough. I would write something like >Every software has issues; so know your software's gotchas. With that said, the perception I get is that PostgreSQL has less such gotchas and works better "out of the box". Though still not optimally, according to our Ops.
PostgreSQL is pretty good- the default config limits the resource consumption to something you very likely wont want; and things like WAL shipping have some awkward gotchas but it's really stable and quite consistent.
People usually argue against postgresql because it used to be slow on multi-core systems, the vacuum was kinda crappy in PGSQL It's like the OpenBSD of Databases.
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#69Earlier quoted context omitted.
While I’m contemplating why people might have thought utf8mb3 was wise in 2004, I’ll ask a similar question that I’d love to know the answer to, of events from my early childhood: why did UTF-16 gain popularity? I get UCS-2 existing, but once it became evident it wasn’t enough, why make UTF-16 and ruin Unicode with surrogate pairs (which still cause trouble today, leading to things like WTF-8 to make it possible for…
With utf-8 you give up any hope of O(1) string operations. With utf-16 you can get O(1) string operations at a cost of doing them on code points rather than graphemes. If greater than 95% of the strings your language/program will ever see are going to be a single code point that trade off seems worth it. I think a lot of people totally discount the enormous cost that Latin-1 users are paying for CJK (etc) support the…
They'd kind of what happened in python 3.3. The internal representation is flexible, depending on the string contents. It may be just 1byte ASCII, or it may be utf32 if needed: https://www.python.org/dev/peps/pep-0393/
Re: In MySQL, don’t use “utf8”, use “utf8mb4” (2016)
#70Earlier quoted context omitted.
Can't reply to the sibling comment by Chris, but there isn't a "utf8 character set". There's just different encodings of a character set. Even after hunting around after going to the 404 you pointed me to, I still get this: utf8mb4: A UTF-8 encoding of the Unicode character set using one to four bytes per character. utf8mb3: A UTF-8 encoding of the Unicode character set using one to three bytes per character. So I do…
Not sure why the reply link is sometimes missing, but in my experience if you open just that comment (via the “N minutes ago” link) the textbox for replying is there and works. Perhaps the link is omitted to discourage rapid conversation? I fixed the link almost immediately—pesky inclusion of the dot in “.)” --- The SQL specification (I cite SQL-92) speaks of character sets (and uses the keyword CHARACTER SET). That…
> utf8mb3 is an alias for utf8 which was introduced after utf8
I didn't know this - and although I don't use MySQL much these days it good to know, thank you.
> in SQL, each Unicode encoding is a CHARACTER SET....
I see what you mean. That's pretty confusing...
> but in my experience if you open just that comment (via the “N minutes ago” link) the textbox for replying is there and works
It totally does!