Live data from Hacker News

Upgrading Uber's MySQL Fleet

uber.com

171–180 of 215 posts

Re: Upgrading Uber's MySQL Fleet

#171

Earlier quoted context omitted.

> I can tell from a mile away that this is written by ChatGPT / Claude, at least partially. Whilst it may smell of ChatGPT/Claude, I think the answer is actually simpler. Look at the authors of the blog, search LinkedIn. They are all based in India, mostly Bangalore. It is therefore more likely to be Indian English. To be absolutely clear, for absolute avoidance of doubt: This is NOT intended a racist comment. Indian…

Can you elaborate on the last part? What are some stylistic patterns that are different when something is written by a US author v/s Indian?

> What are some stylistic patterns that are different when something is written by a US author v/s Indian?

Largely as @brk above you already mentioned, tendency to use formal and obscure words alongside a specific tone. I'll also re-iterate what @brk said, hard to fully describe, more of a "you know it when you see it".

If I had to pick some specific examples from the blog post, the following phrase is a good example:

We systematically advanced through each tier, commencing from tier 5 and descending to tier 0.

There are 101 ways you could write that in US English, but I reckon 99% of the US population would be unlikely to pick the above unless they were writing an academic paper or something.

This one is also quite Indian English in many respects:

Our automated alerts and monitoring system actively oversees the process to ensure a seamless transition and promptly alerts of any issues that may arise.

Similarly, we have stylistic elements such as the over-breaking of paragraphs to the extent it becomes a series of statements. For example:

Upgrading to MySQL 8.0 brought not only new features, but also some unexpected tweaks in query execution plans for certain clusters. This resulted in increased latencies and resource consumption, potentially impacting user experience. This happened for the cluster which powers all the dashboards running at Uber. To address this issue, we collaborated with Percona, identified a patch fix, and successfully implemented it for the affected clusters. The resolution ensured the restoration of optimized query performance and resource efficiency in alignment with the upgraded MySQL version.

A relatively short paragraph, but five phrases. Your average US English writer would likely word it differently resulting in it being trimmed down to two or three phrases.

As I said in my original post though, none of it is bad English, its just a different style.

Re: Upgrading Uber's MySQL Fleet

#172
post #154
post #91

Impressive numbers at a glance but that boils down to ~140qps which is between one and two orders of magnitude below what you'd expect a normal MySQL node typically would serve. Obviously average execution time is mostly a function of the complexity of the query but based on Uber's business I can't really see what sort of non-normative queries they'd run at volume (e.g. for their customer facing apps). Uber's infra r…

Dividing the fleet QPS by the number of nodes is completely meaningless because it assumes that queries are distributed evenly across every part of the system and that every part of the system is uniform (e.g. it is unclear what the read/write patterns are, proportion of these nodes are read replicas or hot standbys, if their sizing and configuration are the same). That isn't realistic at all. I would guess it is ext…

Simple enough just to think about localities and time of day. New York during Tuesday rush hour could be more load than all of North Dakota sees in a month. Even busy cities probably drop down to nothing on a weekday at 3am.

Re: Upgrading Uber's MySQL Fleet

#173

Earlier quoted context omitted.

Let's delve into why you think that

It's simple. Human writing is short and to the point (either because they're lazy or want to save the reader's time), yet still manages to capture your attention. AI writing tends to be too elaborate and lacks a sense of "self". I feel like this article challenges my patience and attention too much, there is really no need to focus on the pros of upgrading here. We reader just want to know how they managed to upgrade…

> Human writing is short and to the point (either because they're lazy or want to save the reader's time)

Good human writing is short and to the point. (Technical writing at least.) But this is not a result of laziness — it’s actually more difficult.

“If I had more time, I would have written a shorter letter.” - Blaise Pascal, and probably others [0]

In any case I find these LLM “gotcha” comments incredibly tedious.

[0] https://quoteinvestigator.com/2012/04/28/shorter-letter/?amp...

Re: Upgrading Uber's MySQL Fleet

#174

Earlier quoted context omitted.

MySQL stores table data in a b+ tree where updates modify the data directly in place as transactions are committed, and overwritten data is moved to a secondary undo log to support consistent reads. MySQL indexes store primary keys and queries rely on tree traversal to find the row in the b+ tree, but it can also contain references to rows in the undo log. PostgreSQL tables are known as heaps, which consist of slotte…

> The upside is Postgres doesn't have to worry about page splits, so things like bulk inserts can be much more efficient. Not in the heap, but if you have any index on the table (I know, don’t do that for bulk loads, but many don’t / it isn’t feasible sometimes) then you’re still dealing with a B+tree (probably). Also, MySQL still gets the nod for pure bulk load speed via MySQLShell’s Parallel Import Utility [0]. You…

> then you’re still dealing with a B+tree

Absolutely, though they're generally orders of magnitude smaller than the table file unless you're INCLUDE'ing lots of columns.

There's pg_bulkload which supports parallel writers as well as deferred index updates until the loading process is complete. Not sure how it compares to what MySQL offers out of the box, but I definitely agree that the MySQL tooling ecosystem in general has a leg up.

Re: Upgrading Uber's MySQL Fleet

#177
post #103

It's sort of funny how can you immediately tell it's LLM sanitized/rewritten.

I got that feeling as well. In addition, I suspect it was originally written for an internal audience and adapted for the 'blog because the references to SLOs and SLAs don't really make sense in the context of external Uber customers.

Re: Upgrading Uber's MySQL Fleet

#179
post #102

Earlier quoted context omitted.

See, the problem is that the people who care about cost performance and the people who care about UX performance are rarely the same people, and often neither side is empowered with the data or experience they need to bridge the gap.

Hardware is cheap relative to salaries. It might take 1 engineer 1 quarter to optimize. Compare that to a few thousand per server.

Ok but we're in a thread about Ubers cloud bills, which are probably well into the 9 figures annually. It definitely gets talked about in board meetings.

Global public cloud spend is hundreds of billions of dollars a year. I wouldn't be surprised if it's AWS's marketing team that came up with the talking point about how much more expensive developer time is.

Edit: put this another way- wherever you work, you might know what parts of the architecture need some performance work but do you know what parts of the architecture cost the most money?

Re: Upgrading Uber's MySQL Fleet

#180
post #164

Earlier quoted context omitted.

Only FULL takes a serious lock (normal vacuum only takes a weak lock preventing things like other vacuums or table alterations iirc). Aside: I wish Postgres forced to make explicit the lock taken. Make me write “TAKE LOCK ACCESS EXCLUSIVE VACUUM FULL my_table”, and fail if the lock I take is too weak. Implicit locks are such a massive footgun that have caused countless incidents across the world, it’s just bad design…

`TAKE LOCK ACCESS EXCLUSIVE VACUUM FULL` is just an incantation that will be blindly copy-pasted. I don't see how it would stop anyone from shooting themselves in the foot.

Imagine two footguns. One shoots your foot off when you open the window. The second requires you to point a gun at your foot and pull the trigger before the window unlocks. Far fewer people will suffer accidental foot injuries from the latter.
Post reply on HN