Before we sold WinJunkie.com we had: - 75K users - 5K daily w/ 10K burst on Sundays - Over 1M db transactions - Rails - Redis - Memcache Heroku: $279 - 400 / month
What does it take to run a web app with 5K – 10K users?
91–100 of 104 posts
Re: What does it take to run a web app with 5K – 10K users?
#92Good database mechanics is key. That is the most important thing in my opinion. That is really the whole point in rails when you are deciding relationships. The abstraction in Rails when deciding what should be the best model structure is the same thing as deciding what should be the best and most efficient table structure in your database.
The rave about MongoDB is that it (maybe not quote me on this) "cures" the need for the desire multi-dimensional database. However, even with MontoDB's ability to expand due to it's not needing a pre-defined structure and the ability to expand out dimensionally to a certain extent, PostgreSQL (claims anyways) is still more efficient if you correctly index your tables (think about how you will be querying) and create the correct relationships. Build out models. Allow flexibility.
Also, don't forget caching. Redis and performing jobs is key in certain situations. However, don't get caught up in too much hype. Especially those coming from closed source technologies (Not just talking about caching technologies here but everything in general). They will sell and produce an atmosphere of necessity, but do some research first. Don't follow the heard. I am not going to call anyone out on this. Just do the research and think why is that necessary. I've mentioned Redis a few times and maybe that isn't even necessary either.
Most importantly, put your stuff out there. If it crashes, so what! At least you know you have something. And then you will have people who will give you advice in a coherent direction if necessary.
I salute you in your efforts. Now the most important part is put it out there and kick ass!
You may even want to look into Redis which is a cache system.
Re: What does it take to run a web app with 5K – 10K users?
#93Well, I think good indexing is key. How do you have your indexes. If they are aligned with what is frequently queried then you should be in good shape. A database of 5-10K users isn't difficult these days for any major DBMS. The key lies in the other things attached to the users and items you query. That can run into the millions. Even that is not much, but depending on how many times you query a million row table, i…
Eventually you will want to maybe bring on other developers. Have a good private repo on Github. That is a choice of course. This way you can choose what to merge into your live branches. You just need to find a good groove. Git is beautiful. It really does help in organization of your development process.
And analytics. I'm a bit absent on this. The reason why is that you are a developer. You will eventually know what you want to know and even if you need to bring on another developer due to time constraints and such, you will know how to develop out personalized analytics that you want out of your app. I mean it is almost evolution. You build an app. Then you want to know what's up. You know the app the best so you will develop loggin and analytical methods to provide you with what you want to know. So again. Unless you are looking at a gem that will help you with visualizing what you are going to develop on the analytical side of things, don't get overwhelmed. You know best. You built it. It's beautiful, and if your app is really a hit, you will plug in chug and make it work.
Re: What does it take to run a web app with 5K – 10K users?
#94"Scaling problems rule . . . "
http://www.youtube.com/watch?v=0CDXJ6bMkMY&t=23m34s
Sounds like you have the skills to grow this up to 500 to 1k paying customers on your own. Early on you can increase ram/cpu to handle any initial scaling issues.
Once you even have 1k customers you'll have revenue to hire experts to help you with scaling and security.
Good luck in 2015.
Re: What does it take to run a web app with 5K – 10K users?
#95Re: What does it take to run a web app with 5K – 10K users?
#962. Do not invest time and/or money in learning another programming language or framework until you are sure that for a specific component of your product, programming language X will perform at least 2 times better with 2 times less HW resources.
3. Stressing again on the app stack (I saw some really pushy comments on changing the programming language), it is rarely the bottleneck of a web app. You'll scale your storage stack way earlier and more often than the app stack.
4. Know your data. That's how you decide if it's better to use a RDBMS, document store, k/v store, graph database etc. Like I said before, you're going to scale your data before any other layer becomes a problem so choosing the right data storage solution is crucial. Don't be afraid to test various storage solutions. They usually have good -> great documentation and ruby tends to be a good friend to every technology. There's a gem for everything. :)
5. Scale proportionate to your business/product growth. You will have to scale at some point. But be careful to scale proportionate to your growth. For example, if the number of users will double, get the hardware that suffices that growth. Less HW resources will lead to a slower user experience thus user dissatisfaction. More HW resources than needed will increase your costs and the resources that are not needed will stay unused. Why waste money?!
These are my 2c. As your business gets bigger - I hope it does - other problems will occur. But usually these things will last up to 100k users.
Disclaimer: this is for a generic web app as you didn't give us any details. Depending on the app, some of my points might be inaccurate or invalid.
Re: What does it take to run a web app with 5K – 10K users?
#97The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
Re: What does it take to run a web app with 5K – 10K users?
#98The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…
> * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. +1 You can't log too much. The user who claims an important email never arrived - does your system say it was sent? This bug 3 users have reported yet no one can reproduce - what were they doing at the time and wh…
Re: What does it take to run a web app with 5K – 10K users?
#99Earlier quoted context omitted.
Logging every hit will always require a lot of space. But there are some tricks you can use to "compress" it: hash long strings like the URL and user agent and store the hash as binary instead of a string. A 100+ byte string can compress to just 16 or 32 bytes depending the hash your pick. Store the hash lookup in a separate table.
what is the benefit of your approach? The lookup table will still have data growth issues
Re: What does it take to run a web app with 5K – 10K users?
#100Earlier quoted context omitted.
I had never even heard of your site. Could it be you need to do a little marketing?
True, it started as a hobby, so it’s been fully bootstrapped and I’ve spent less than $2500 on actual marketing to date. Advertising in the Health / Diet space is super pricy, the ROI still doesn’t make sense. CPA >>>>>> LTV