Before we sold WinJunkie.com we had:
- 75K users
- 5K daily w/ 10K burst on Sundays
- Over 1M db transactions
- Rails
- Redis
- Memcache
Heroku: $279 - 400 / month
What does it take to run a web app with 5K – 10K users?
31–40 of 104 posts
Re: What does it take to run a web app with 5K – 10K users?
#32The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
good list.
I just want to add more voice to the unit testing part. It's natural to feel that testing will slow down the development process , but once you got some unit tests for the core parts of the app, it feels awesome and the developers will have a lot of confidence in the product.
Re: What does it take to run a web app with 5K – 10K users?
#33I have a little over 10k DAU for http://cronometer.com/ and it runs on 2 fairly small cloud instances + a cloud database. A single host can handle the load fine, but I keep 2 running to survive failures.
And it took me nearly 4 years to get that many users. We can’t all grow like facebook!
Re: What does it take to run a web app with 5K – 10K users?
#34Earlier quoted context omitted.
> * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. +1 You can't log too much. The user who claims an important email never arrived - does your system say it was sent? This bug 3 users have reported yet no one can reproduce - what were they doing at the time and wh…
Getting loads of core services out into third parties is really wonderful for logging. E.g. if email sending happens in Mandrill, then you never need to write decent logging calls for that and you have a reliable source of truth!
Except you won't know if your server ever sent it to Mandrill. :) Always be extremely verbose with logging!
Re: What does it take to run a web app with 5K – 10K users?
#35Are you talking about 5-10k simultaneous users? It's very subjective depending on the usage profile of your site. The best thing is to pick a service like AWS/Heroku/Cloud Hosting Service that allows you to grow your offering. After you get to a point of profitability, you can look at greater efficiencies like setting up your own hosting hardware depending on your needs, if that makes sense. Our company has grown to a point where we are looking at implementing our own multi-datacenter cloud to save on hosting fees. But that's a good problem to have :D
Re: What does it take to run a web app with 5K – 10K users?
#36The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> Rails should take care of most of this for you I often found myself falling into the "I'm not using PHP, so I don't have to worry about any security holes" trap. CSRF is something you really need to watch out for if you are constructing forms manually!
No need to construct forms manually, Rails form helpers take care of authenticity tokens automatically.
Re: What does it take to run a web app with 5K – 10K users?
#37Earlier quoted context omitted.
StackExchange does 590million monthly views with around half that [1] and it includes redundancy. Do you think 24 could be overkill? [1] http://stackexchange.com/performance#
Not overkill as servers are not equal and different apps require different infrastructure. The bulk of my servers are small worker servers that go out and fetch feeds. Parallel downloads, high network throughput, low CPU/memory.
[deleted]
Re: What does it take to run a web app with 5K – 10K users?
#38The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
good list. I just want to add more voice to the unit testing part. It's natural to feel that testing will slow down the development process , but once you got some unit tests for the core parts of the app, it feels awesome and the developers will have a lot of confidence in the product.
TDD speeds up development, especially when adding features. Without full test coverage, you're relying upon manual regression testing to catch bugs. Once you have actual users, that could be catastrophic to deploy broken code without even realizing it.
Re: What does it take to run a web app with 5K – 10K users?
#39The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> Backups - Take regular backups of all user data. It's also VERY important that you actually try restoring the data as well, as it's quite possible that backups are not occurring properly. The part about testing your backups is huge. I can't count how many projects I've been on that had problems where we needed to restore and we looked only to find any number of problems. Oh, backups actually stopped last month when…
And don't use keys on your console admin accounts.
Re: What does it take to run a web app with 5K – 10K users?
#40The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. +1 You can't log too much. The user who claims an important email never arrived - does your system say it was sent? This bug 3 users have reported yet no one can reproduce - what were they doing at the time and wh…
I'd actually argue it is possible to log too much if you aren't using good tools to make your logs easily searchable. Which is why you should use such tools if at all possible. Otherwise the logs can become so big that finding the entries for that bug or that email becomes pretty much impossible. This is also why it's important to take a few minutes and think about what you're logging and how. Things like request and user IDs can be invaluable. My test is usually "if I have nothing but this error message/log entry, do I have enough to begin investigating?". This is hard to get right until a bug or problem occurs and you actually need to use the logs but investing a bit of time into it can be a life saver.