Live data from Hacker News

What does it take to run a web app with 5K – 10K users?

news.ycombinator.com

41–50 of 104 posts

Re: What does it take to run a web app with 5K – 10K users?

#41
Obviously this depends a lot on the kind of usage patterns your application provides, but in my experience you should be able to comfortably run a 30k _daily_ users (the total n. of registered users could be 10x-100x that, depends) web app on a single $200-300/mo AWS machine (with hourly payments, you can get a better rate by prepaying). Get a bunch of cores and plenty of memory and put both your web app and the DB server on the same box for ultimate simplicity. Postpone scaling efforts for as long as humanly possible, probably until PMF and accelerating growth.

With reasonably "low level" tooling such as Java/Clojure/Haskell/whatever and a properly configured Postgres instance you should be able to go quite far. You're very unlikely to be CPU-challenged in the web app (again, no idea what your web app is going to be doing, so it's just a guess), most of the memory and CPU will be consumed by your database server caching and running queries. You should be able to handle a good 500-1000 db transactions / sec without much hassle.

IMHO most of the challenge will be making something that 10k people will want to use daily, not actually being able to scale to that many users.

Re: What does it take to run a web app with 5K – 10K users?

#42
post #34
post #26

Earlier quoted context omitted.

Getting loads of core services out into third parties is really wonderful for logging. E.g. if email sending happens in Mandrill, then you never need to write decent logging calls for that and you have a reliable source of truth!

Except you won't know if your server ever sent it to Mandrill. :) Always be extremely verbose with logging!

I have always heard the opposite, that too much logging is as bad as no logging. I see the point of having the logs to be able to find out what happened, but what happens when there' s so much logging that the information needed is just buried into huge amount of noise?

Re: What does it take to run a web app with 5K – 10K users?

#44
post #2

The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…

> * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space.

Depending on the service you're building, you can log too much. Consider the privacy and security implications of the existence of those logs; anything you log can be subpoenaed, but logs that don't exist cannot be.

Consider anonymizing your logs from day 1, and only turning on non-anonymous logging upon a report from a user. Alternatively, give users a "report a problem" button, and save their last N minutes of otherwise-ephemeral logs only when they hit that button.

You absolutely want to log enough to help you debug the service, but do you really need to archive old logs, or should you delete them entirely?

Re: What does it take to run a web app with 5K – 10K users?

#45
post #2

The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…

also i think one should take care of Reliability into account as well, if you have only one app server it will fail - maybe eventually - and you'll find yourself trying to ssh into the server at 2am, have at least a few app servers load balanced, have 2 db servers replicating with auto failover etc..

Re: What does it take to run a web app with 5K – 10K users?

#46
post #3

I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly. My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, wor…

Hey that "queue for free account" thing is brilliant. Good presentation too!

Re: What does it take to run a web app with 5K – 10K users?

#47
post #2

The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…

> Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space.

How do most people manage activity logs? Currently what we have set up is the user id (if the user is logged in), IP address, URL they hit, user agent, and timestamp are all inserted into an activity logs table. For one particular site with an API that's being polled the size of the DB grew pretty large.

Re: What does it take to run a web app with 5K – 10K users?

#48
post #3

I run Minimal Reader [1] (which has about 5k users) and I can say that running a web app like this takes a lot of time and energy keeping everything running smoothly. My service has a lot of moving parts, all of which are distributed among a couple dozen different servers. Keeping the technical infrastructure running smoothly requires a lot of data visualization of server stats, database stats, web request stats, wor…

Hey that "queue for free account" thing is brilliant . Good presentation too!

I agree, I haven't seen the "Wait in line for a free account" before. Can you describe this feature in more detail? For example, how long does a user have to wait, does it allow in more users as the size of the queue grows, etc.?

Re: What does it take to run a web app with 5K – 10K users?

#49
post #2

The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time, one of my side projects with 50K users was hosted in a EC2 small instance. With that out of the way, here are a few things you will need to take care of: * Security (especially passwords) - Rails should take care of most of this for you, but you should ensure t…

> Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. How do most people manage activity logs? Currently what we have set up is the user id (if the user is logged in), IP address, URL they hit, user agent, and timestamp are all inserted into an activity logs table. For…

Logging every hit will always require a lot of space. But there are some tricks you can use to "compress" it: hash long strings like the URL and user agent and store the hash as binary instead of a string. A 100+ byte string can compress to just 16 or 32 bytes depending the hash your pick. Store the hash lookup in a separate table.
Post reply on HN