Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

goodfil.ms

Post reply on HN
Print view View original

21–30 of 42 posts

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#21

Post by purephase »

zhoutong wrote: post #15

This is almost exactly how I deployed NameTerrific ( https://www.nameterrific.com ). It automatically gets ready for any horizontal scaling in the future. In addition, I allocate the workers to their own instances so that they can grow independently. Also I use Capistrano to automate deployment. Whenever additional resources are needed, simply clone the instance and add it to the load balancer. I use 2 Nginx workers…

If you don't mind me asking, what load balancer are you using? I'm trying to setup multiple app servers on EC2 behind ELB using Redis as a session_store and I can't get sessions to persist.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#22

Post by dumbluck »

Nice post, but:

Don't self-back-pat.

It read more like marketing for the company than anything of much worth.

Don't use the words "web scale".

It is a meaningless term. How many requests per second and how many guest and user sessions for how long of a time is "web scale"? Does "web scale" just mean Christmas shopping traffic on your crappy e-commerce site that no one visits, does it mean you can survive being top link on HN on a Friday morning, or slashdotted, or DoS attacked by anonymous, or survive a fucking flood and the data center lifts off the ground and enough flexibility and strength in the trunk lines to handle a tsunami?

Don't just throw users at it.

Unless testing is very costly and you need every user's eyes on your untested code as possible, that is just stupid. Look at Tsung, Grinder or JMeter, or the many other ways you could generate load as a first step before you do that.

Don't gloss over the details.

Sure you said you were using Rails 3.2 and postgres and a tad bit about the architecture, but who in the hell doesn't know that you need to load balance, need to put the DB servers on different VMs/servers than the apps. Although- having everything on both and some stuff just not turned on and live is not a bad idea for emergency fault tolerance, and you didn't mention that.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#23

Post by riffraff »

zhoutong wrote: post #15

This is almost exactly how I deployed NameTerrific ( https://www.nameterrific.com ). It automatically gets ready for any horizontal scaling in the future. In addition, I allocate the workers to their own instances so that they can grow independently. Also I use Capistrano to automate deployment. Whenever additional resources are needed, simply clone the instance and add it to the load balancer. I use 2 Nginx workers…

> Uses MySQL or Postgres as the datastore, deployed to a single instance, with frequent data backups to cloud storage

this does not seem to be "ready for horizontal scaling in the future".

I don't know whether you have multiple slaves, and TFA itself says that they'd scale the database _vertically_, I'm just stating that there's a pretty important mismatch between what you say and what the article says.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#24

Post by damncabbage »

camwest wrote: post #17
instakill wrote: post #14

Earlier quoted context omitted.

To answer my own question, there is a free addon called pgbackups that will do this for you, and Heroku automatically load balances between dynos.

Pgbackups is not for load balancing. That is already automatic. Pgbackups is for managing Postgres backups.

3. Anyone have a link to a run down of how to back up your Postgres DB periodically?

"Pgbackups" answered this question.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#25

Post by marknutter »

dumbluck wrote: post #22

Nice post, but: Don't self-back-pat. It read more like marketing for the company than anything of much worth. Don't use the words "web scale". It is a meaningless term. How many requests per second and how many guest and user sessions for how long of a time is "web scale"? Does "web scale" just mean Christmas shopping traffic on your crappy e-commerce site that no one visits, does it mean you can survive being top li…

Your caps-lock key appears to be broken.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#26

Post by dumbluck »

marknutter wrote: post #25
dumbluck wrote: post #22

Nice post, but: Don't self-back-pat. It read more like marketing for the company than anything of much worth. Don't use the words "web scale". It is a meaningless term. How many requests per second and how many guest and user sessions for how long of a time is "web scale"? Does "web scale" just mean Christmas shopping traffic on your crappy e-commerce site that no one visits, does it mean you can survive being top li…

Your caps-lock key appears to be broken.

Oh, you're right- thanks! Just fixed the comment. I'll get the capslock key fixed on Monday.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#27

Post by subpixel »

zhoutong wrote: post #16
instakill wrote: post #13

A few questions: 1. Is it possible to spin up two apps on Heroku? 2. What load balancers are available with the above? 3. Anyone have a link to a run down of how to back up your Postgres DB periodically? Thanks

For 1 and 2: AFAIK Heroku already has three layers, the Nginx routing mesh, Varnish cache and your dynos. Everything is already handled for you so if you have two dynos, the traffic will rotate between the two. You don't need a load balancer for Heroku. It's built-in.

Note: Varnish isn't available on the (standard) Cedar stack. Here's how Heroku suggests you approach cacheing:

https://devcenter.heroku.com/articles/http-caching

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#28

Post by zhoutong »

purephase wrote: post #21
zhoutong wrote: post #15

This is almost exactly how I deployed NameTerrific ( https://www.nameterrific.com ). It automatically gets ready for any horizontal scaling in the future. In addition, I allocate the workers to their own instances so that they can grow independently. Also I use Capistrano to automate deployment. Whenever additional resources are needed, simply clone the instance and add it to the load balancer. I use 2 Nginx workers…

If you don't mind me asking, what load balancer are you using? I'm trying to setup multiple app servers on EC2 behind ELB using Redis as a session_store and I can't get sessions to persist.

I used to use ELB, but later I migrated the whole app to a self-hosted private cloud on a dedicated server.

I'm using Nginx as the load balancer, and there're no session problems for me because I use the built-in secure cookie store.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#29

Post by zhoutong »

riffraff wrote: post #23
zhoutong wrote: post #15

This is almost exactly how I deployed NameTerrific ( https://www.nameterrific.com ). It automatically gets ready for any horizontal scaling in the future. In addition, I allocate the workers to their own instances so that they can grow independently. Also I use Capistrano to automate deployment. Whenever additional resources are needed, simply clone the instance and add it to the load balancer. I use 2 Nginx workers…

> Uses MySQL or Postgres as the datastore, deployed to a single instance, with frequent data backups to cloud storage this does not seem to be "ready for horizontal scaling in the future". I don't know whether you have multiple slaves, and TFA itself says that they'd scale the database _vertically_, I'm just stating that there's a pretty important mismatch between what you say and what the article says.

Well, it's true that database isn't horizontal-scaling-ready as described in the OP. But for many web apps, the database is not the bottleneck. Efforts should be spent on memory caching and NoSQL long before vertical scaling generates diminishing returns.

Of course, it's easy to add a master-slave replication to a single database instance when the need arises, even possible without downtime. So I assume most RDBMSs to be horizontal-scaling-ready.

The difficult part is the app stack. We need to be able to deploy more instances in minutes, not hours or days. If at the beginning we cram everything (Nginx, Unicorn and MySQL) into a single dedicated server it can be really difficult to increase the capacity in the future without downtime or additional expense. Instead, even if you don't need the power of several dedicated servers, it's worthwhile to virtualize a single server and split into a few boxes and make them horizontal-scaling-ready. Then you can clone, resize and migrate the instances.

This is what I do to increase the capacity of my app in a few minutes:

  1. Create a new instance, and load the pre-built app image
  2. Assign an IP address and add it to the DNS (like app-3.nameterrific.net pointing to an internal IP)
  3. Add the app server to deploy.rb (for Capistrano)
  4. cap deploy
  5. Add the IP to load balancer
That's it. This stack can be used from day one affordably and it can scale to potentially billions of visitors.

Re: The Minimum Viable Rails Stack, Now Reddit Frontpage Approved

#30

Post by smharris65 »

zhoutong wrote: post #29
riffraff wrote: post #23

Earlier quoted context omitted.

> Uses MySQL or Postgres as the datastore, deployed to a single instance, with frequent data backups to cloud storage this does not seem to be "ready for horizontal scaling in the future". I don't know whether you have multiple slaves, and TFA itself says that they'd scale the database _vertically_, I'm just stating that there's a pretty important mismatch between what you say and what the article says.

Well, it's true that database isn't horizontal-scaling-ready as described in the OP. But for many web apps, the database is not the bottleneck. Efforts should be spent on memory caching and NoSQL long before vertical scaling generates diminishing returns. Of course, it's easy to add a master-slave replication to a single database instance when the need arises, even possible without downtime. So I assume most RDBMSs t…

>But for many web apps, the database is not the bottleneck.

Really? I think the database is usually the bottleneck in most web apps. That's why you cache data: to avoid hitting the database.

Post reply on HN