A few years ago I was inspired by this artice: http://viniciusvacanti.com/2010/11/01/6-things-you-need-to-learn-to-build-your-own-prototype/. Now I am able to build my own Rails apps and run them in production. I am now wondering what it takes to launch and successfully run a website with 5-10K users? All the articles I have found doing a web search were vary basic.
What does it take to run a web app with 5K – 10K users?
1–10 of 104 posts
Re: What does it take to run a web app with 5K – 10K users?
#2The most difficult thing is going to be getting to 10K active users :)
These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of:
* Security (especially passwords) - Rails should take care of most of this for you, but you should ensure that you patch vulnerabilities when they are discovered. Also, stuff like having only key-based login to your servers etc.
* Backups - Take regular backups of all user data. It's also VERY important that you actually try restoring the data as well, as it's quite possible that backups are not occurring properly.
* One click deployment - Use Capistrano or Fabric to automate your deployments.
* A good feedback/support system - this could even be email to begin with (depending on the volume you expect), but it should be accessible.
* Unit tests - as your app grows in complexity, you will never be able to test all the features manually. I'm not a big fan of test driven development, but really, start writing unit tests as soon as you have validated your product idea.
* Alerts, monitoring and handling downtime - Downtimes are inevitable. Your host or DNS could go down, you might run out of disk space, etc. Use something like Pingdom to alert you of such failures.
* Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space.
Re: What does it take to run a web app with 5K – 10K users?
#3I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly.
My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, worker stats, user stats, etc. I have everything piped into a nice dashboard so we can see if there is anything odd happening at a glance. When things break (and they will) you need to know where to look first.
Having 5k users also requires time to help them with support issues. Users generate a lot of bug reports, questions, and suggestions. To keep paying users happy, I offer a 1-day response time on support issues, which requires me to spend quite a bit of time sending emails.
Then, of course, if you want to grow the app, you need to spend time marketing it. We could talk for hours about this.
The list goes on and on. Feel free to shoot me an email (email in my profile) if you want to talk specifics about anything.
Re: What does it take to run a web app with 5K – 10K users?
#4The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
That's a good overview. I would also not forget about the type of application. Like you said, resources are pretty cheap these days, maybe leveraging Amazon cloud and integrating with other applications could life the heavy weight that is normally required. If someone can get to 10k active users, they should also note on possible scaling issues down the line.
Re: What does it take to run a web app with 5K – 10K users?
#5I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly. My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, wor…
24 servers? Can you elaborate on your architecture a little more? What tools do you use to help you manage them?
Re: What does it take to run a web app with 5K – 10K users?
#6I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly. My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, wor…
24 servers? Can you elaborate on your architecture a little more? What tools do you use to help you manage them?
One server to serve an RSS reader to the 5k users, 23 others to host 23 different stats and monitoring applications to make sure that one server is working OK?
Re: What does it take to run a web app with 5K – 10K users?
#7The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> Rails should take care of most of this for you
I often found myself falling into the "I'm not using PHP, so I don't have to worry about any security holes" trap. CSRF is something you really need to watch out for if you are constructing forms manually!
Re: What does it take to run a web app with 5K – 10K users?
#8Could you clarify if its 5-10k users concurrent? per month? per day?
Re: What does it take to run a web app with 5K – 10K users?
#9The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space.
+1 You can't log too much. The user who claims an important email never arrived - does your system say it was sent? This bug 3 users have reported yet no one can reproduce - what were they doing at the time and what else was going on?
No, I'm not at that stage yet (of effectively being able to rewind application state in the log files to see what was going on), but for debugging issues in production it's exceedingly useful.
Re: What does it take to run a web app with 5K – 10K users?
#10I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly. My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, wor…
This looks like a really slick app - congratulations! I'm actually working on an RSS reader at the moment with a few similar philosophies (respecting privacy, lightweight design etc.). There's a lot of competition in the RSS reader space since the sad demise of Google Reader - how did you market the site and get your first users?