Server for 100,000+ users on site at once • Hacker News Board

Re: Server for 100,000+ users on site at once

#11

Post by StatusStalker » Wed, Mar 30, 2011, 4:46 AM UTC

asharp wrote: post #6

StatusStalker wrote: post #4

Earlier quoted context omitted.

watching videos (embedded from other sites), listening to music, uploading photos, and sharing data. Our test site is running so slow now and it only loads 20 activities at a time.

How is your site structured? 1) You want to move static data off your system onto another site like s3. 2) You want to do whatever you can async. ie. if somebody shares data, you don't have to make it available immediately. Just place it in a queue and have a second system go through and deal with it.

1. Yes 2. Yes

People submit data and it's displayed, 20 activities a time and if all of those activities are videos it is taking about 17 seconds to load

Re: Server for 100,000+ users on site at once

#12

Post by StatusStalker » Wed, Mar 30, 2011, 4:49 AM UTC

bmelton wrote: post #7

StatusStalker wrote: post #4

Earlier quoted context omitted.

watching videos (embedded from other sites), listening to music, uploading photos, and sharing data. Our test site is running so slow now and it only loads 20 activities at a time.

How is your test suite running? Are you running 20 instances of your site on a single test machine? Is it possible that the machine is the bottleneck? While the machine is doing its 20 tests, can you load the site another, unburdened machine?

I think part of the problem is we are loading embedded videos instead of thumbnails as it's taking about 17 seconds to load the page with 20 videos now. We have a semi basic machine through staminus.net, but I really want to get my own server that can handle 100,000+ users ay a time. I haven't considered having the load on another unburdened machine but that's a really good idea.

Re: Server for 100,000+ users on site at once

#13

Post by StatusStalker » Wed, Mar 30, 2011, 4:52 AM UTC

Yaa101 wrote: post #8

StatusStalker wrote: post #4

Earlier quoted context omitted.

watching videos (embedded from other sites), listening to music, uploading photos, and sharing data. Our test site is running so slow now and it only loads 20 activities at a time.

At least make sure you have a fairly modern machine (dedicated, not shared) of round 2GHz, a 100MHz link to the net instead of 10MHz and have Nginx/php-fpm instead of Apache webserver installed. This should take you a long way to your minimum requirements. If this does not do the trick you probably have to cluster or cloud several machines. Clustering is more barebones to the machine but harder to install and finetun…

Thanks for the advice! Checking my SQL is a really good idea, I really do need to reorganize the database.

Re: Server for 100,000+ users on site at once

#14

Post by asharp » Sun, Apr 03, 2011, 3:10 PM UTC

asharp wrote: post #6

StatusStalker wrote: post #4

Earlier quoted context omitted.

watching videos (embedded from other sites), listening to music, uploading photos, and sharing data. Our test site is running so slow now and it only loads 20 activities at a time.

How is your site structured? 1) You want to move static data off your system onto another site like s3. 2) You want to do whatever you can async. ie. if somebody shares data, you don't have to make it available immediately. Just place it in a queue and have a second system go through and deal with it.

Why is it taking 17 seconds to load, if the page you are serving them merely contains links?