Is the new implementation still Rails?
No, it's still Ruby but built directly on top of Rack.
How do you know?
How Shopify reduced storefront response times with a rewrite
11–20 of 76 posts
Re: How Shopify reduced storefront response times with a rewrite
#12Some of the listed optimizations were:
> We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries.
> Reducing Memory Allocations
> Implementing Efficient Caching Layers
All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custom Lua/nginx routing and two applications instead of a monolith).
Shopify surely has tons of Rails experts and I assume they pondered a lot before going for this unusual rewrite, so of course they have their reasons, but I really didn't understand (from the article) what they accomplished here that they couldn't have done in the Rails monolith.
You don't need to ditch Rails if you just don't want to use ActiveRecord.
Re: How Shopify reduced storefront response times with a rewrite
#13Re: How Shopify reduced storefront response times with a rewrite
#14Is the new implementation still Rails?
That’s also my question after reading this post. When trying to shave off milliseconds by going for a full rewrite, moving away from ruby seems like an obvious decision...at least intuitively..
Obvious how?
Are you going to restructure literally thousands of employees and their teams, staffed with Rubyists and organized around your current setup?
Will you re-hire and/or re-train everyone?
That doesn't seem so obvious... At the scale of a team like Shopify, refactoring to a different language is probably a non-starter.
Re: How Shopify reduced storefront response times with a rewrite
#15Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…
[deleted]
Re: How Shopify reduced storefront response times with a rewrite
#16Earlier quoted context omitted.
No, it's still Ruby but built directly on top of Rack.
How do you know?
Because he apparently works at Shopify?
Re: How Shopify reduced storefront response times with a rewrite
#17Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…
Someone replied but deleted right when I was posting this answer, so I'm replying to myself:
What I didn't understand was why the listed performance optimizations couldn't be implemented in the monolith itself and ensued the development of a new application, which is still Ruby.
In a production env, the request reaches the Rails controller pretty fast.
I know for a fact that the view layer (.html.erb) can be a little slow if you compare it to, say, just a `render json:`, but if you're still going to be sending fully-rendered HTML pages over the wire, the listed optimizations (caching, query optimization and memory allocation) could all be implemented in Rails itself to a huge extent, and that's what I'd love to know more about.
Re: How Shopify reduced storefront response times with a rewrite
#18Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…
(contributor here)
The project does still use code from Rails. Some parts of ActiveSupport in particular are really not worth rewriting, it works fine and has a lot of investment already.
The MVC part of Rails is not used for this project, because the storefront of Shopify works in a very different way than a CRUD app, and doesn’t benefit nearly as much. Custom code is a lot smaller and easier to understand and optimize. Outside of storefront, Shopify still benefits a lot from Rails MVC.
I’ll also add that storefront serves a majority of requests made to Shopify but it’s a surprisingly tiny fraction of the actual code.
Re: How Shopify reduced storefront response times with a rewrite
#19This is very interesting. N+1 and lazy loading have been a very common problem that profilers can spot, but eager loading also has a cartesian product problem where if you have an an entity with 6 sub item, and 100 of another subitem, you'll end up getting 600 rows to construct a single object / view model.
I have been recently playing with RavenDB (from my all time favorite engineer turned CEO), it approaches most of these as an indexing problem in the database, where the view models are calculated offline as part of indexing pipeline. It approaches the problem from a very pragmatic angle. It's goal is to be a database that is very application centric.
Still to be seen if we will end up adopting, but it'll be interesting to play with.
Disclaimer: I am a former NHibernate contributor, and have been very intimate with AR features and other pitfalls.
Re: How Shopify reduced storefront response times with a rewrite
#20Some of the listed optimizations were: > We carefully vet what we eager-load depending on the type of request and we optimize towards reducing instances of N+1 queries. > Reducing Memory Allocations > Implementing Efficient Caching Layers All of those steps seem pretty standard ways of optimizing a Rails application. I wished the article made it clearer why they decided to pursue such a complex route (the whole custo…
(contributor here) The project does still use code from Rails. Some parts of ActiveSupport in particular are really not worth rewriting, it works fine and has a lot of investment already. The MVC part of Rails is not used for this project, because the storefront of Shopify works in a very different way than a CRUD app, and doesn’t benefit nearly as much. Custom code is a lot smaller and easier to understand and optim…
> because the storefront of Shopify works in a very different way than a CRUD app
Any interesting/successful patterns you can share/resources you can share on said patterns?