Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Shopify reduced storefront response times with a rewrite

engineering.shopify.com

Post reply on HN
Print view View original

21–30 of 76 posts

Re: How Shopify reduced storefront response times with a rewrite

#21

Post by nicoburns »

sbarre wrote: post #14
bsaul wrote: post #3

Earlier quoted context omitted.

That’s also my question after reading this post. When trying to shave off milliseconds by going for a full rewrite, moving away from ruby seems like an obvious decision...at least intuitively..

Obvious how? Are you going to restructure literally thousands of employees and their teams, staffed with Rubyists and organized around your current setup? Will you re-hire and/or re-train everyone? That doesn't seem so obvious... At the scale of a team like Shopify, refactoring to a different language is probably a non-starter.

If you have thousands of rubyists then you surely have hundreds who also know other languages? Seems to make sense to use a fast langauage for the small performance sensitive part of your codebase.

Re: How Shopify reduced storefront response times with a rewrite

#22

Post by cookiecaper »

sbarre wrote: post #14
bsaul wrote: post #3

Earlier quoted context omitted.

That’s also my question after reading this post. When trying to shave off milliseconds by going for a full rewrite, moving away from ruby seems like an obvious decision...at least intuitively..

Obvious how? Are you going to restructure literally thousands of employees and their teams, staffed with Rubyists and organized around your current setup? Will you re-hire and/or re-train everyone? That doesn't seem so obvious... At the scale of a team like Shopify, refactoring to a different language is probably a non-starter.

Yeah. Consider that BigCos end up writing transpilers and new runtimes for their target platforms before rewriting the application, which would entail discarding the decades of built-in bugfixes and application logic as well as reconstructing the organization around a different platform -- HipHop for PHP, Grumpy, etc. A language change is no small thing in any company of appreciable size.

Re: How Shopify reduced storefront response times with a rewrite

#23

Post by balfirevic »

tehlike wrote: post #19

This is very interesting. N+1 and lazy loading have been a very common problem that profilers can spot, but eager loading also has a cartesian product problem where if you have an an entity with 6 sub item, and 100 of another subitem, you'll end up getting 600 rows to construct a single object / view model. I have been recently playing with RavenDB (from my all time favorite engineer turned CEO), it approaches most o…

Didn't NHibernate have the cartesian product problem solved in a neat way by having various fetch strategies?

You could specify to eagerly load some collections and have NHibernate issue additional select statement to load the children, producing maximum of 2-3 queries (depending on the eager-loading depth) but avoiding both N+1 problem and cartesian row explosion problem.

Re: How Shopify reduced storefront response times with a rewrite

#24

Post by notsureaboutpg »

Most commenters are focused on the optimizations made, but I actually think the custom routing and verification mechanism is the interesting bit.

That kind of a tool could be handy in lots of scenarios (comparing the same service written in two different languages or with different dependencies, etc).

But how does their verifier mechanism deal with changes in the production database between responses? If the response of the legacy service comes first and the response of the new service comes after, in between both responses (the request being the same) couldn't the data from the database change and thus result in the responses not passing verification when they otherwise should have? How do they manuever around that issue?

Great write-up by the way! I really liked it :)

Re: How Shopify reduced storefront response times with a rewrite

#25

Post by tehlike »

balfirevic wrote: post #23
tehlike wrote: post #19

This is very interesting. N+1 and lazy loading have been a very common problem that profilers can spot, but eager loading also has a cartesian product problem where if you have an an entity with 6 sub item, and 100 of another subitem, you'll end up getting 600 rows to construct a single object / view model. I have been recently playing with RavenDB (from my all time favorite engineer turned CEO), it approaches most o…

Didn't NHibernate have the cartesian product problem solved in a neat way by having various fetch strategies? You could specify to eagerly load some collections and have NHibernate issue additional select statement to load the children, producing maximum of 2-3 queries (depending on the eager-loading depth) but avoiding both N+1 problem and cartesian row explosion problem.

yes, that's the common method, but you still end up issuing multiple network calls. The problem wit issuing select statements to load the children is you have to wait on the first query (root) to finish so you can issue others which adds to the network latency (usually low, but it also depends). It's still not as good as having materialized viewmodels on server where you can issue a single query to get everything you need. The disadvantage is the storage cost, though.

Re: How Shopify reduced storefront response times with a rewrite

#26

Post by pushrax »

joshmn wrote: post #20
pushrax wrote: post #18

Earlier quoted context omitted.

(contributor here) The project does still use code from Rails. Some parts of ActiveSupport in particular are really not worth rewriting, it works fine and has a lot of investment already. The MVC part of Rails is not used for this project, because the storefront of Shopify works in a very different way than a CRUD app, and doesn’t benefit nearly as much. Custom code is a lot smaller and easier to understand and optim…

> because the storefront of Shopify works in a very different way than a CRUD app Any interesting/successful patterns you can share/resources you can share on said patterns?

“Not a CRUD app” isn’t a design decision, it’s just that storefront is almost entirely read-only, and the views are merchant-provided Liquid code. Most of a shop's data can be accessed on any page; data dependencies are in large part defined by the view, not the controller.

Re: How Shopify reduced storefront response times with a rewrite

#27

Post by tekstar »

joshmn wrote: post #20
pushrax wrote: post #18

Earlier quoted context omitted.

(contributor here) The project does still use code from Rails. Some parts of ActiveSupport in particular are really not worth rewriting, it works fine and has a lot of investment already. The MVC part of Rails is not used for this project, because the storefront of Shopify works in a very different way than a CRUD app, and doesn’t benefit nearly as much. Custom code is a lot smaller and easier to understand and optim…

> because the storefront of Shopify works in a very different way than a CRUD app Any interesting/successful patterns you can share/resources you can share on said patterns?

Shopify's storefront is based around a liquid renderer instance. If you look up how objects are added to the liquid context that is pretty similar to the overall pattern (or at least was back when I worked there, hi pushrax :)

Re: How Shopify reduced storefront response times with a rewrite

#28

Post by Jach »

nicoburns wrote: post #21
sbarre wrote: post #14

Earlier quoted context omitted.

Obvious how? Are you going to restructure literally thousands of employees and their teams, staffed with Rubyists and organized around your current setup? Will you re-hire and/or re-train everyone? That doesn't seem so obvious... At the scale of a team like Shopify, refactoring to a different language is probably a non-starter.

If you have thousands of rubyists then you surely have hundreds who also know other languages? Seems to make sense to use a fast langauage for the small performance sensitive part of your codebase.

Seems also that since Ruby is not going to be taught as part of people's normal formal education in programming, you can expect Rubyists to be on average more capable of... learning new things.

So yes, "re-train". Give everyone a book on the new language, maybe pay for some online courses from pluralsight or wherever, cancel meetings for a week. You can learn a lot faster than in a school environment when you've got paid 8 hour days to put into a single subject + coworkers to chat with.

Besides, it's not like they don't get to avoid learning new things anyway, even if you restrict it to the Ruby ecosystem. In the JS world (which I'm sure they all know too, as one tends to when working on web sites even if you're mostly back-end) as new revisions of the language come out people have to keep up with the syntax and changing idioms.

"For some reason, programmers love to learn new stuff, as long as it's not syntax." -- Steve Yegge

Re: How Shopify reduced storefront response times with a rewrite

#29

Post by nthj »

pqdbr wrote: post #17
pqdbr wrote: post #12

Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…

Someone replied but deleted right when I was posting this answer, so I'm replying to myself: What I didn't understand was why the listed performance optimizations couldn't be implemented in the monolith itself and ensued the development of a new application, which is still Ruby. In a production env, the request reaches the Rails controller pretty fast. I know for a fact that the view layer (.html.erb) can be a little…

They talk about reducing memory allocations. My guess is the rest of the app is very large and they’re benefiting from not sharing memory and GC with that.

Of course, everything you said is true for a small-to-medium sized Rails application.

They likely could have explored a separate Rails app to meet this goal, but then they have to maintain the dependency tree and security risks twice. And if the Rails core refactors away any optimizations they make, they have to maintain and integrate with those.

There’s definitely some wiggle room and a judgement call here but their custom implementation has merit.

Re: How Shopify reduced storefront response times with a rewrite

#30

Post by adrr »

pushrax wrote: post #18
pqdbr wrote: post #12

Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…

(contributor here) The project does still use code from Rails. Some parts of ActiveSupport in particular are really not worth rewriting, it works fine and has a lot of investment already. The MVC part of Rails is not used for this project, because the storefront of Shopify works in a very different way than a CRUD app, and doesn’t benefit nearly as much. Custom code is a lot smaller and easier to understand and optim…

If it’s rail, can you just expose the rails cache object in liquid? Give control to merchants. That would yield bigger speed improvements.
Post reply on HN