File under "things you will never need to do if you use cloud services".
Nah random outages because the RDS instance you were on decided to faceplant itself, or the weird memory to bandwidth scaling AWS has chosen will make you pull your hair out on a high traffic day. It's just different problems.
Upgrading Uber's MySQL Fleet
121–130 of 215 posts
Re: Upgrading Uber's MySQL Fleet
#122Wait until they find out they have to upgrade to 8.4 now
And also all the passwords away from mysql_native_password
Re: Upgrading Uber's MySQL Fleet
#123It's sort of funny how can you immediately tell it's LLM sanitized/rewritten.
It reads like any of those tech blogs, using big words where not strictly necessary but also not wrong Don't know about your LLM feeling
Also this paragraph sounds a lot like it has been written by LLMs, it's over-expressive:
We systematically advanced through each tier, commencing from tier 5 and descending to tier 0. At every tier, we organized the clusters into manageable batches, ensuring a systematic and controlled transition process. Before embarking on each stage of the version upgrade, we actively involved the on-call teams responsible for each cluster, fostering collaboration and ensuring comprehensive oversight.
The paragraph uses "commencing from" together with "descending to". People would probably write something like "starting with". It shows how the LLM has no spatial understanding: tier 0 is not below or above tier 5, especially as the text has not introduced any such spatial ordering previously. And it gets worse: there is no prior mention of the word "tier" in the blog post. The earlier text speaks of stages, and lists 5 steps (without giving them any name, but the standard term is more like "step" instead of "tier").There is more signs like "embark", or that specific use of "fostering collaboration" which goes beyond corporate-speak, it also sounds a lot like what an LLM would say. Apparently "safeguard" is also a word LLMs write very often.
Re: Upgrading Uber's MySQL Fleet
#124Earlier quoted context omitted.
Didn't realize their entire MySQL data layer runs in AWS. Given that they went with basically a blue-green update strategy, this was, essentially a "witness our cloud spend" kind of post.
They're not. Almost all of their infra was on prem when I worked there 3 years ago.
Re: Upgrading Uber's MySQL Fleet
#125I wonder if an upgrade like this would be less painful if the db layer was containerized? The migration process they described would be less painful with k8s. Especially with 2100+ nodes/VMs
Re: Upgrading Uber's MySQL Fleet
#126Earlier quoted context omitted.
It reads like any of those tech blogs, using big words where not strictly necessary but also not wrong Don't know about your LLM feeling
It contains the word "delve", a word that got way more popular in use since the introduction of LLMs. Also this paragraph sounds a lot like it has been written by LLMs, it's over-expressive: We systematically advanced through each tier, commencing from tier 5 and descending to tier 0. At every tier, we organized the clusters into manageable batches, ensuring a systematic and controlled transition process. Before emba…
Re: Upgrading Uber's MySQL Fleet
#127Having spent a couple months doing a corporate mandated password rotation on our services - a number of which weren't really designed for password rotation - happy to see the dual password thing mentioned. Being able to load in a new password while the current one is active is where it's at! Trying to coordinate a big bang where everyone flips over at the same time is misery, and I spent a bunch of time updating serv…
This isn't 100% equal as ownership (thus permissions with DEFINER) in stored procedures etc. needs some thought, but bad access using outdated username is simpler to trace (as username can be logged etc. contrary to passwords; while MySQL allows for tracing using performance_schema logging incl. user defined connection attributes which may ease finding the "bad" application)
Re: Upgrading Uber's MySQL Fleet
#128File under "things you will never need to do if you use cloud services".
Re: Upgrading Uber's MySQL Fleet
#129It's sort of funny how can you immediately tell it's LLM sanitized/rewritten.
Let's delve into why you think that
I feel like this article challenges my patience and attention too much, there is really no need to focus on the pros of upgrading here. We reader just want to know how they managed to upgrade at that large scale, challenges they faced and how the solved them. Not to mention any sane tech writers that value their time wouldn't write this much.
Re: Upgrading Uber's MySQL Fleet
#130It's sort of funny how can you immediately tell it's LLM sanitized/rewritten.
It reads like any of those tech blogs, using big words where not strictly necessary but also not wrong Don't know about your LLM feeling