Live data from Hacker News

We replaced Redis with MySQL for inventory reservations and it scaled

shopify.engineering

161–170 of 282 posts

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#161

Why even have a blog when you can't be arsed to write the posts. This is so obviously LLM-written. I have a positive view of Shopify engineers, but this kind of made a dent in that confidence.

I agree with you, it doesn't take long to identify the itemized style that LLMs use and it's always incredibly annoying.

I would rather read the original thoughts of the engineer, grammar mistakes and stylistic imperfections included, than prompt generated AI slop.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#162
post #159

Why even have a blog when you can't be arsed to write the posts. This is so obviously LLM-written. I have a positive view of Shopify engineers, but this kind of made a dent in that confidence.

What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site. As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writ…

People see an — dash and kneejerk. Writers are deciding to stop using this valid punctuation because of this exact reaction, it's ridiculous.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#163
post #162
post #159

Earlier quoted context omitted.

What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site. As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writ…

People see an — dash and kneejerk. Writers are deciding to stop using this valid punctuation because of this exact reaction, it's ridiculous.

A true casualty of AI. Em-dashes used to be one of my favorite punctuation devices, now I find myself consciously editing them out of my writing.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#164
post #159

Why even have a blog when you can't be arsed to write the posts. This is so obviously LLM-written. I have a positive view of Shopify engineers, but this kind of made a dent in that confidence.

What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site. As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writ…

Because the post is just rambling without a clear intent or direction. Why do you need "oversell protection" if you have "transactions". Isn't the whole point of a "transaction" that it handles concurrency and disk failures?

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#165
post #159

Why even have a blog when you can't be arsed to write the posts. This is so obviously LLM-written. I have a positive view of Shopify engineers, but this kind of made a dent in that confidence.

What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site. As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writ…

Subheading and dot point spam, low density writing (the opposite of standard technical english), including useless detail (like enumerating stats on Shopify's scale), using contrastive parallelism, and other llm-isms. Even if it's not AI it's bad writing done by someone who has picked up AI's worst ticks.

For example this subheading:

> "The real bottleneck: connections, not CPU"

That's two AI smells. AI likes to say vacuous punchy statements like "the final takeaway" or "here's the rub". Then contrastive parallelism "connections, not CPU". Contrastive parallelism should scarcely exist in technical writing, regardless of whether it's AI generated.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#166

Earlier quoted context omitted.

I agree, it does seem awfully complicated and there are quite a few pieces missing for this to be a complete solution. I'm a bit surprised about the scalability case against a simpler solution. This is not about Shopify's scale. We're talking about contention for a specific SKU of a specific seller at a specific warehouse location. How many shopping carts are competing for a single SKU at the payment stage at peak ho…

Flash sales are a huge scaling issue for Shopify. There are celebrities who want to sell thousands of items in a few minutes window at the end of an advertised countdown. Basically this is an incredibly rare case but a feature that they want to support.

There were woot offs 20 years ago. Flash sales are not some new scaling issue.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#167

Could not they shard the inventory table by shop_id? As I understand, the order includes only items from one store, so there is no need to keep all the stores in a single table. Also, I wonder why they could not have a row status (available/reserved) and UPDATE it instead of deleting the rows.

They never said they don’t shard it, however this doesn’t solve the problem they were facing. Even if they have a single store (therefore a single shard), the burst demand may be high for the item in that shop, which creates contention for “remaining item quantity” resource. Their solution spreads this contention across several rows.

> Also, I wonder why they could not have a row status (available/reserved) and UPDATE it instead of deleting the rows.

This requires a row per item unit, doesn’t it? If you have 50k units you’ll have to track status of every item, meaning 50k rows. They also mention this as a rationale to use at most 1k rows, and treat it as a buffer.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#168

Earlier quoted context omitted.

> not the best design [...] So those engineers at Shopify worked hard for months on a more performant system, but they missed the obvious structure? They chose a complex denormalization for no good reason? It may be true, but I think it's presumptuous to belittle their work when we have only partial information. My guess is that they had good reasons to think that the more obvious ways would not scale. And from readi…

i also work in big tech and know that a lot of bullshit design creeps into system design and prod, because everyone is overworked, overstressed, wants to just get things done for the quarterly performance review as to not get shitcanned with severance re concurrency, it is not a big issue at all. stock exchanges deal with HFT traders and can easily deal with concurrency of orders. Same can be implemented with shopify…

> re concurrency, it is not a big issue at all

I would really appreciate it if you could write this up as an article. It would be an extremely interesting and valuable read

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#169
post #159

Earlier quoted context omitted.

What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site. As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writ…

Subheading and dot point spam, low density writing (the opposite of standard technical english), including useless detail (like enumerating stats on Shopify's scale), using contrastive parallelism, and other llm-isms. Even if it's not AI it's bad writing done by someone who has picked up AI's worst ticks. For example this subheading: > "The real bottleneck: connections, not CPU" That's two AI smells. AI likes to say…

I really wish that slop writing was disincentivized in whatever RLHF they do. I don’t want to read the weird LinkedIn pop-sci tone for the rest of my life in such amounts.

I wonder if the average reader is also getting annoyed like this or whether they just don’t care - especially seeing what seems to get upvoted on your run of the mill social media sites. They probably collectively shape things more than I do.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#170

not the best design to have 1000 rows for each shop*SKU combination. If a candidate proposed this solution during Shopify's System Design interview, i doubt he would be vetted for Senior+ position. Instead of having 1000 rows per shop*SKU, why not just have one row per shopping cart*SKU? That way a single row would represent a single cart, and will hold info of multiple items of the same SKU. No need a cludge with 10…

Your mental model here is mapping too close to an actual cart in a retail, at a in person, setting.

The assumption that a SKU maps 1 to 1 to a cart item is flawed.

If the first item in the cart is a bundle of SKU-A and SKU-B, the second item is a bundle of SKU-A and SKU-C and the third item is 5xSKU-B where do you do you keep the re-agregation of the SKU-X's to track them?

This is without accounting for item location in the reservation - and rules that may apply around that.

You haven't even gotten to the part where different customers will have different rules around shipping from different locations - because that can eat into margins.

You're also making a bunch of other assumptions around transaction flow and where carts are actually stored (and how they get converted to an invoice, with payment attached) that likely do not hold true.

Could you do it more like what you're sugesting -- maybe -- but only in a single tenant system.

Post reply on HN