In case you want to convert your MySQL database tables from utf8 to utf8mb4 please remember that each character will now take 4, not 3 bytes. This means you can easily run out of limit for a row length (65536 bytes) if you have many string columns and your indexes will grow. Also, if a column is a part of a foreign key reference you might have to remove the foreign key first, upgrade columns on both sides and add the foreign key back.
So converting an existing database might be not as easy as it seems before.
This bit me hard many years ago in possibly the most frustrating bug I've encountered--frustrating because of the stupidity of MySQL's implementation. I had a utf8 MySQL database that was created back before utf8mb4 existed--this is, back in the days when emojis were just getting popular, but not quite popular enough to be on software keyboards; and when only a few people had smartphones and iPads, but not everyone.…
I think the lesson is "don't silently fail," and in that regard mysql has failed at learning that lesson over and over again to the point where the lesson is now, "don't use mysql" because what else can you do when dealing with such shitty software? This is far, far from the only stupid bug like this. Use mysql only if you don't care about preserving data. Period. There are many valid use cases that don't care about preserving data 100%, apparently most companies, so it's not a huge issue in reality. Hopefully the data that deals with monetary transactions won't get corrupted, but when it does, one cannot complain if they used mysql. It's that simple.
Oh. In that case, you do mean ASCII, methinks (which also works), with Latin-1 you're shooting yourself in both feet. (Looks for link to "falsehoods programmers believe about encodings")
To my understanding, ASCII basically only covers English and Italian well, whereas Latin-1 covers most Western European languages — which includes the main languages of the Americas (Spanish, English, French, Portuguese).
> To my understanding, ASCII basically only covers English and Italian well
And German, too: while German does have ä, ö, ü & ß, they may correctly be written as ae, oe, ue & ss.
Possibly my favourite thing about the rise of emoji is that they’re not Basic Multilingual Plane, and so stupid hacks like MySQL’s utf8 character set (I seriously don’t know why anyone ever thought it was a good idea in 2002–2004) are now obviously insufficient for full Unicode support, and enough people care about them that adoption of this basic level of Unicode is driven, and so non-English speakers are inadverten…
Combined emoji, such as 🇺🇸, are great for this as well! Things like also help - separating them out, even on spaces, breaks the emoji. We are really making progress towards proper processing.
This bit me hard many years ago in possibly the most frustrating bug I've encountered--frustrating because of the stupidity of MySQL's implementation. I had a utf8 MySQL database that was created back before utf8mb4 existed--this is, back in the days when emojis were just getting popular, but not quite popular enough to be on software keyboards; and when only a few people had smartphones and iPads, but not everyone.…
I think the lesson is "don't silently fail," and in that regard mysql has failed at learning that lesson over and over again to the point where the lesson is now, "don't use mysql" because what else can you do when dealing with such shitty software? This is far, far from the only stupid bug like this. Use mysql only if you don't care about preserving data. Period. There are many valid use cases that don't care about…
does that affect MariaDB as well? That would only leave people to PostgreSQL or NoSQL Databases like MongoDB AFAIK. I've never been a fan of MySQL, but MariaDB always seemed to me like the spark of hope that was left in there.
Oh my god, this is both terrifying and could one day prove to be the most valuable comment I've seen on HN. I love MySQL, but the fact that they didn't make utf8mb4 the default for mysqldump (instead of utf8), when they created utf8mb4 for database use... is one of the most irresponsible and negligent pieces of engineering I've seen in a long time, considering the amount of data worldwide backed up with mysqldump. Th…
Imagine how many companies out there are relying on backups that are already corrupted... is your company one of them?
This is an important reason why testing your backups is a critical part of backing up.
> If you need a database, don’t use MySQL or MariaDB. Use PostgreSQL. This is stupid. There are pros and cons to every database. For example, MySQL allows you to set a trigger on an op, PostgreSQL requires you to write a function first.
What is so stupid with avoiding the living hell of maintaning crap: silent data truncation, weird inconcistencies or this very issue.
If you need a database, why would you ever pick the stupid one..,
I think the lesson is "don't silently fail," and in that regard mysql has failed at learning that lesson over and over again to the point where the lesson is now, "don't use mysql" because what else can you do when dealing with such shitty software? This is far, far from the only stupid bug like this. Use mysql only if you don't care about preserving data. Period. There are many valid use cases that don't care about…
does that affect MariaDB as well? That would only leave people to PostgreSQL or NoSQL Databases like MongoDB AFAIK. I've never been a fan of MySQL, but MariaDB always seemed to me like the spark of hope that was left in there.
Yes because it's built on the same codebase with the same design decisions and bugs. Maybe in the future it will diverge enough and fix such issues although compatibility with mysql might be affected.
One factor is that back in the early 90s China was still an economically backward, closed Communist enclave barely more integrated into the global economy than North Korea is now. One of the deciding factors in having to decisively move beyond the BMP was China's adoption of GB 18030 as a mandatory standard in 2006[1]. [1] https://en.wikipedia.org/wiki/GB_18030
> One factor is that back in the early 90s China was still an economically backward, closed Communist enclave barely more integrated into the global economy I'd disagree, China was well on the way to becoming an economic power house in the early 90's: https://en.wikipedia.org/wiki/Five-year_plans_of_China#Eight...
But it wasn't yet. Decades later, the infrastructure stabilized (although in 2010 consumer network products were still incredibly chaotic).
This bit me hard many years ago in possibly the most frustrating bug I've encountered--frustrating because of the stupidity of MySQL's implementation. I had a utf8 MySQL database that was created back before utf8mb4 existed--this is, back in the days when emojis were just getting popular, but not quite popular enough to be on software keyboards; and when only a few people had smartphones and iPads, but not everyone.…
I already had substantial beef with MySQL and took a job I wanted despite the fact we were using it. Eight months later we are fixing exactly this bug.
In the world of production code issues most people rank data loss as the most severe or the second most severe category of error.
Designing this kind of crap into a database is just pissing on your users. It was the last straw. I can’t trust you if you work like this. I’m not working with any code written by these clowns again.