Making a site that can handle #1 on Hacker News • Hacker News Board

Re: Making a site that can handle #1 on Hacker News

#41

Post by saurik » Mon, Apr 29, 2013, 5:35 AM UTC

tahoecoder wrote: post #39

paulsutter wrote: post #37

Earlier quoted context omitted.

Surely there's an opportunity here. It's mind boggling how people are having performance issues here, but they are. It's an opportunity for someone to make some money and improve the internet, by fixing whatever tools they're using. To amplify your comment, processors today process billions of instructions per second. Even if all 3000 pageviews _did_ hit within one minute, thats hundreds of millions of instructions a…

Lots of people are misunderstanding that chartbeat figure. It's not 287 visits in a 45 minute window. These are visitors who are currently interacting with your site. Some are idle but most aren't.

Can you give us a more useful understanding of how many pageviews your site actually had? You only have three webpages. If we assume that for each visit the user visited all three of them and then further reloaded your home page twenty times, that's still fewer than one pageview per second.

As you had a spike to 282 concurrent visitors (four times your average), even under that unrealistic amount of reloading, that's less than four requests per second. (Again, really: one way to look at that figure is "in a 45 minute window"; I provide the broken-out math below to make it clearer.)

Re: Making a site that can handle #1 on Hacker News

#42

Post by saurik » Mon, Apr 29, 2013, 5:45 AM UTC

tahoecoder wrote: post #40

saurik wrote: post #38

Earlier quoted context omitted.

You had ~2250 visits that day (1440 minutes), each of which spent (on average) ~45 minutes looking at your website. Assuming only one concurrent visitor, you'd have had 1440/45 = 32 total visits. You actually had 2250/32 = ~70 times that number, so you averaged ~70 concurrent visits that day. Due to expected fluctuations (day/night, when you posted to HN) there will be considerable real-time variation; it is easy to…

Creating a daily average of concurrent visits isn't relevant. Servers don't take an average of all the possible future hours of complete inactivity when they are serving up requests. The post was only up at the top for a couple of hours before the server performance got so bad such that everyone started flagging it. My post then dropped off the front page and quickly went to page 4 or 5 due to the flagging. I agree w…

> Servers don't take an average of all the possible future hours of complete inactivity when they are serving up requests.

I did that math to demonstrate that you are misunderstanding the relevancy of the Chartbeat number: that yes, if you look at the values "over 45 minutes" that is slightly weird, but entirely accurate, as it is not in any way unlikely that you had a spike four times your average during the day (that's why I had to calculate the average: to see if your maximum value was unexpected; I did the calculation of the average using the way of interpreting the chart which you believe is flawed, and it turns out that it is entirely consistent).

> The post was only up at the top for a couple of hours before the server performance got so bad such that everyone started flagging it.

Sure: your site was slow (I'm not questioning that your site was slow: tons of websites hit HN and then fall apart, it seems to be a daily occurrence), but I still don't know how you accomplished it with this small a number of users.

Re: Making a site that can handle #1 on Hacker News

#43

Post by kennywinker » Mon, Apr 29, 2013, 5:50 AM UTC

zalew wrote: post #24

kennywinker wrote: post #9

While the sentiment is clear, I had an uncached Wordpress site on shared hosting withstand #2 or #3 (I forget where exactly it peaked). HN isn't all that huge a traffic deliverer. It's just about the quality of that traffic.

> HN isn't all that huge a traffic deliverer. It's just about the quality of that traffic. what?

What jacques_chester said, except I was mostly thinking about the engagement level of the traffic. I.e. I go to HN for deep reads and to find out about cool new tools and services. When I go to reddit, for example, I'm there to lol at teh funny .jpgs.

Re: Making a site that can handle #1 on Hacker News

#44

Post by chacham15 » Mon, Apr 29, 2013, 6:01 AM UTC

I dont understand what the problem is with using a slow backend even for static content (just because it is easier to program). Just use varnish to cache the pages and you're good, right? Or am I missing something?

Re: Making a site that can handle #1 on Hacker News

#45

Post by eagsalazar2 » Mon, Apr 29, 2013, 6:17 AM UTC

tahoecoder wrote: post #15

baby wrote: post #14

You have no dynamic content and your website crash at 200~ concurrent users? You're doing everything wrong then. I had a website going through 6000 concurrent users sometimes and which was hosted on a very cheap mutualised server! I didn't realize so many people had no idea about simple caching techniques.

The issue is a static site shouldn't even need to use caching techniques. When I'm building an app I will use memcached with redis. But this post was just about getting a host environment/workflow for situations in which you want the ease of development that server side languages provide (shared code includes, etc.) and not have to deal with things like caching.

This is your #1 most hilarious comment.

Re: Making a site that can handle #1 on Hacker News

#46

Post by romain_dardour » Mon, Apr 29, 2013, 8:34 AM UTC

Actually you can combine middleman and dynamic pages to get fast static pages and still keep a few dynamic endpoints. We did this on our website http://hull.io for email registration, and blogged about it here : http://blog.hull.io/post/45912703356/the-perfect-almost-stat... - when 90% of your users only consume static content, you greatly benefit from this.

Re: Making a site that can handle #1 on Hacker News

#47

Post by baby » Mon, Apr 29, 2013, 4:57 PM UTC

eagsalazar2 wrote: post #45

tahoecoder wrote: post #15

Earlier quoted context omitted.

The issue is a static site shouldn't even need to use caching techniques. When I'm building an app I will use memcached with redis. But this post was just about getting a host environment/workflow for situations in which you want the ease of development that server side languages provide (shared code includes, etc.) and not have to deal with things like caching.

This is your #1 most hilarious comment.

I guess that's what happens when people learn a lot of technologies in school and don't really know when to use them.

Re: Making a site that can handle #1 on Hacker News

#48

Post by tedchs » Mon, Apr 29, 2013, 8:55 PM UTC

If only there was a way for young adults to learn how to be respectful and understand "time and place" for various behaviors. Oh yeah, there is, parenting.