How Shopify reduced storefront response times with a rewrite • Hacker News Board

Re: How Shopify reduced storefront response times with a rewrite

#61

Post by throwdbaaway » Fri, Aug 21, 2020, 8:26 AM UTC

hevelvarik wrote: post #54

>An example of these foundations is the decision to design the new implementation on top of an active-active replication setup. As a result, the new implementation always reads from dedicated read replicas, improving performance and reducing load on the primary writers. Could someone please explain how the ‘as a result’ follows from the active-active replication setup?

Based on the comment from pushrax, it looks like this is just circular async replication between the old writer and the new writer. For some reason, the old implementation had to send both read and write traffic to the old writer, while the new implementation can do proper read-write split, by reading from dedicated read replicas hanging off the new writer (again, via async replication).

Due to power law, ecommerce generally benefits a lot from things like caching and read-write split. Reading between the lines, it feels like shopify may not yet have sufficient experience in dealing with async replication, and all the potential issues caused by replication lag. Fun time ahead.

Re: How Shopify reduced storefront response times with a rewrite

#62

Post by pushrax » Fri, Aug 21, 2020, 8:35 AM UTC

tekstar wrote: post #27

joshmn wrote: post #20

Earlier quoted context omitted.

> because the storefront of Shopify works in a very different way than a CRUD app Any interesting/successful patterns you can share/resources you can share on said patterns?

Shopify's storefront is based around a liquid renderer instance. If you look up how objects are added to the liquid context that is pretty similar to the overall pattern (or at least was back when I worked there, hi pushrax :)

Yep, the main idea is to set up the liquid interpreter with the right variables/methods and the right liquid templates, and evaluate the result. There’s a lot of code that runs around that, but the path-specific code is quite small.

Hello :)

Re: How Shopify reduced storefront response times with a rewrite

#63

Post by bradfeehan » Fri, Aug 21, 2020, 8:35 AM UTC

switch11 wrote: post #59

can anyone add to that article data on What users saw in terms of response time and perceived response time And what users are seeing after the improvements * We had evaluated spotify for one of our projects and aesthetically it is really good. However, time wise their store takes forever to do stuff This was a couple of years back, so hopefully things are much better now Basically, the article covers how much better…

spotify?

Re: How Shopify reduced storefront response times with a rewrite

#64

Post by dx034 » Fri, Aug 21, 2020, 8:45 AM UTC

Scarbutt wrote: post #53

bdibs wrote: post #47

I’m aware that Ruby/Rails isn’t that quick, but it seems mind boggling that an 800ms server response time is considered tolerated, and 200ms is satisfying. I’ve never used Ruby in production so maybe my reference point is off and this is more impressive than I’m giving it credit for.

For page reloads, anything below 300ms is fine.

But you should also account for up to 100-200ms network latency (especially with mobile networks) plus some rendering time. A 200ms server response time can already lead to a perceived 500ms loading time.

Re: How Shopify reduced storefront response times with a rewrite

#65

Post by MirrorNext » Fri, Aug 21, 2020, 9:41 AM UTC

gravypod wrote: post #45

Shopify has traditionally been an example people have pointed to for scaling a monolith with a large growth factor in all areas: team size, features, user base size, general "scale" of the company. Does anyone on here, who has worked on this project or internally at Shopify, feel that this project was successful? Do you think this is the first, of a long and gradual process, where Shopify will rewrite itself into a m…

At Shopify, we build into the monolith unless there’s a strong reason to build it as a new service.

It makes more sense for us to extract things than to make everything microservice.

Storefront makes sense to be on its own service, so we are making it so.

Re: How Shopify reduced storefront response times with a rewrite

#66

Post by spondyl » Fri, Aug 21, 2020, 9:54 AM UTC

I'd be interested to know if setting Service Level Objectives were considered as an alternative to using Apdex? Given that it's nice to be able to then calculate an error budget out of your SLO and use that to determine whether changes were impacting to the customer experience or not. Well, so the theory goes anyway. Actually doing it in practice is a whole different story ;)

Re: How Shopify reduced storefront response times with a rewrite

#67

Post by gadders » Fri, Aug 21, 2020, 10:42 AM UTC

The bit I found interesting in this is how they compare and verify that two web pages rendered by different methods "match".

I wonder how you would do that? You can't hash the html. Do you take screenshots and compare?

Re: How Shopify reduced storefront response times with a rewrite

#68

Post by chiefalchemist » Fri, Aug 21, 2020, 10:59 AM UTC

vlovich123 wrote: post #49

chiefalchemist wrote: post #41

Earlier quoted context omitted.

It's not only software. 1) What is the goal? What defines success? 2) What are the KPI's? How are we going to measure it? These are baseline questions to any endeavor of substance. Yet, they are rarely defined.

It’s also important to remember that not everything worth doing or every “success” state you set can have KPIs defined (either actually impossible or the science may not be there yet).

To clarify, I was using KPI in the abstract. That is, how do I/we define success? What does it look like? How will we know if we are or we're not?

Re: How Shopify reduced storefront response times with a rewrite

#69

Post by jashmatthews » Fri, Aug 21, 2020, 11:13 AM UTC

nicoburns wrote: post #60

jashmatthews wrote: post #46

Earlier quoted context omitted.

"Faster" languages often have big advantages in small benchmarks which get a lot smaller or even reverse once you're looking at whole application performance. Mandelbrot (from CLBG) Ruby 246s NodeJS 8s Java 4s Web (fortunes from TE benchmarks) Ruby + Roda + Sequel 51k rps NodeJS + Express 46k rps Java + Dropwizard 62k rps

You're comparing Ruby to other options that are still slow: Java (vertx-postgres) 347k rps, Go (fasthttp) 320k rps Rust (actix-postgres) 607k rps

Right but I'm doing that because those are frameworks in other languages which offer a comparable developer experience.

fasthttp isn't even a web framework. It's not surprising that using a raw HTTP library is dramatically faster than using a full framework and ORM but it's also not a sustainable way to build complex web applications with 1000+ developers.

Re: How Shopify reduced storefront response times with a rewrite

#70

Post by nicoburns » Fri, Aug 21, 2020, 11:17 AM UTC

jashmatthews wrote: post #69

nicoburns wrote: post #60

Earlier quoted context omitted.

You're comparing Ruby to other options that are still slow: Java (vertx-postgres) 347k rps, Go (fasthttp) 320k rps Rust (actix-postgres) 607k rps

Right but I'm doing that because those are frameworks in other languages which offer a comparable developer experience. fasthttp isn't even a web framework. It's not surprising that using a raw HTTP library is dramatically faster than using a full framework and ORM but it's also not a sustainable way to build complex web applications with 1000+ developers.

You don't need to have 1000 developers working on the small performance sensitive part of your application though. Split it out into its own application, and then have a small dedicated team.

I can't speak to fasthttp as I haven't used Go much, but actix-web in Rust is a full framework (not as full as something like Rails, but certainly more than mature enough to be used for production projects).