Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced Redis with MySQL for inventory reservations and it scaled

shopify.engineering

Post reply on HN
Print view View original

271–278 of 278 posts

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#271

Post by [deleted] »

codedokode wrote: post #261
jdw64 wrote: post #124

Earlier quoted context omitted.

Does Redis become that slow when you enable both AOF and RDB? Sure, there's a write cost, but it doesn't lose its ability to maintain tens of thousands of connections. Redis supports AOF and lets you choose the fsync policy. But I think using only MySQL is unnecessarily expensive, just to get single transaction tracking for bug tracing. So the article's argument seems to be: 'Use only MySQL as a solution to the distr…

I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, number of records, opens a file, and appends records one by one, making fdatasync after each one. On my SSD, the throughput is 120 fsyncs/second, or ~0.5 MB/s. So if I had Redis to flush the log after each operation, it wouldn't be able to serve those thousands of…

[deleted]

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#272

Post by [deleted] »

codedokode wrote: post #261
jdw64 wrote: post #124

Earlier quoted context omitted.

Does Redis become that slow when you enable both AOF and RDB? Sure, there's a write cost, but it doesn't lose its ability to maintain tens of thousands of connections. Redis supports AOF and lets you choose the fsync policy. But I think using only MySQL is unnecessarily expensive, just to get single transaction tracking for bug tracing. So the article's argument seems to be: 'Use only MySQL as a solution to the distr…

I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, number of records, opens a file, and appends records one by one, making fdatasync after each one. On my SSD, the throughput is 120 fsyncs/second, or ~0.5 MB/s. So if I had Redis to flush the log after each operation, it wouldn't be able to serve those thousands of…

[deleted]

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#273

Post by jdw64 »

codedokode wrote: post #261
jdw64 wrote: post #124

Earlier quoted context omitted.

Does Redis become that slow when you enable both AOF and RDB? Sure, there's a write cost, but it doesn't lose its ability to maintain tens of thousands of connections. Redis supports AOF and lets you choose the fsync policy. But I think using only MySQL is unnecessarily expensive, just to get single transaction tracking for bug tracing. So the article's argument seems to be: 'Use only MySQL as a solution to the distr…

I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, number of records, opens a file, and appends records one by one, making fdatasync after each one. On my SSD, the throughput is 120 fsyncs/second, or ~0.5 MB/s. So if I had Redis to flush the log after each operation, it wouldn't be able to serve those thousands of…

Concurrent connection count and durable write throughput are different things. And AOF doesn't necessarily mean one fsync per record. The benchmark is taking the worst-case speed of Redis and comparing it to a best-case SQL benchmark.

Redis AOF isn't one fsync per record—Redis can use group commit to share fsync across multiple commands. In other words, that could mean a difference of tens of times in your benchmark. I see it differently.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#274

Post by dessimus »

resonious wrote: post #267
dessimus wrote: post #227

Earlier quoted context omitted.

Would a retail business want the inventory to decrement just because users put items into carts that get abandoned? Seems like that would really mess things up more in the long run.

Yes. You decrement for a few minutes then release if there's no purchase in time. Airlines figured out how to do this a very long time ago.

Except airlines, literally oversell flights all the time, and the context of my response was to preventing oversale.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#275

Post by imthatsteve »

lelanthran wrote: post #260
ChrisMarshallNY wrote: post #256

Earlier quoted context omitted.

It's likely because no one could be bothered to respond. There's plenty there, but it's not their job to fetch it for you. People on the Internet (especially here), seem to think that it's OK to make rude demands of complete strangers, and expect them to spend considerable time, meeting them. I'd rather some rando think they "won the Internet," than spend a bunch of my time, working for free, just to satisfy them. I…

> There's plenty there, but it's not their job to fetch it for you. It's been asked about 30 times now. If it existed, someone would have posted it. Maybe you.

Your clearly doing it wrong, on the internet your supposed to make a positive claim such as; No blog exists from the pre-llm days that include the llm-isms. And then you wait for the internet trolls to come out of the woodwork and prove you wrong.

You dont ask people on the internet to provide you with some kinda proof because nobody can be bothered, you dont even ask them to prove you wrong because they cant be bothered. You have to confidently and boldly assert your false claim and that will enrage some poor autists somewhere into doing what you want.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#276

Post by imthatsteve »

cowboylowrez wrote: post #157
ramon156 wrote: post #143

Why even have a blog when you can't be arsed to write the posts. This is so obviously LLM-written. I have a positive view of Shopify engineers, but this kind of made a dent in that confidence.

I just wanted to drop a note that I almost never go into an article or blog wondering if its AI or not. I am trying to be a more picky reader and try to actually do something occasionally but I do browse database articles and this is a good one. So I wonder, am I becoming insensitive about AI writing? Am I being hypnotized into taking whatever color pill thats had a color representing a surrender to the AI "hive mind…

I honestly didnt notice it was ai until after i read the comments. I didnt really care for the article that much since it reminded me of a less knowledgeable person summarizing what a more knowledgeable person said. Like reading an article written by one of those game journalists who hasnt figured out how to use the game controls yet.

I figured it was written by a marketing person on behalf of an engineer or something but after reading comments it was clearly the ai trying to dress up the technical info in some kinda low level appeal to everyone regardless of their technical prowess and that makes no sense to me since non technical people won't care about the article at all at best its basically an ad for people to sell on their platform but im not sure why it would be posted here instead of facebook or twitter or anywhere you might reach the kinda people who sell on shopify.

Instead they took a subject matter that would appeal to the hn audience and then dressed it up in a way that would annoy that same audience. If your going to bother using ai to make your ad pretty you could at least use the ai to target the correct audience in the way they would appreciate.

Re: We replaced Redis with MySQL for inventory reservations and it scaled

#277

Post by nozzlegear »

jasonlotito wrote: post #234
nozzlegear wrote: post #39

Earlier quoted context omitted.

It's not hard to parse, but it's a dense pair of sentences that say nothing. It just pads the length of the article and gives readers mental fatigue trying to read between the lines to figure out what the point is.

> It's not hard to parse That's what I said. Full stop. > but... a bunch of stuff that just pads the comment agreeing with me but adding nothing else of value.

> a bunch of stuff that just pads the comment agreeing with me but adding nothing else of value.

I think you need to "pick up a book." What I wrote disagrees with you, and to think otherwise is just sophistry.

Post reply on HN