Does seem a bit faster -- could be placebo though :)
Why HN was slow and how Rtm fixed it
11–20 of 202 posts
Re: Why HN was slow and how Rtm fixed it
#12Would be interesting to see if traffic goes up after this and is elastic. Marissa Mayer had a talk at some conference in 2009 where she explained her early tests on number of search results on Google - 10, 20, 25, 30 - but in the end it was just about the speed associated with loading the pages that accounted for the number of pageviews and visitors.
Re: Why HN was slow and how Rtm fixed it
#13Sounds like there is a scalability issue within MzScheme in that it iterates over the number of threads, asking each thread about the sockets it has. As one can tell, once # of threads and # of sockets grow - finding which thread to run in user space becomes awfully expensive. As any clever admin will do, a least invasive fix involving limiting the number of connections and threads was done - with what sounds like immediate results!
I have no idea what MzScheme is but I am curious about why is HN running threads in user space in 2011? The OS kernel knows best what thread to pick to run and that is a very well tuned, O(1) operation for Linux and Solaris.
Re: Why HN was slow and how Rtm fixed it
#14Who's Filo? David Filo?
That was my first thought and it would appear so. Filo was a big FreeBSDer
Re: Why HN was slow and how Rtm fixed it
#15Who's Filo? David Filo?
Yep. He's still a mensch. Yahoo did some incredible stuff on Apache & FreeBSD back in the day. I remember a hack that added hardcoded HTTP headers to the image files on disk, to squeeze that extra nilth percent out of the server.
Re: Why HN was slow and how Rtm fixed it
#16HN only supports 20 req per second???
flat files, no database
Re: Why HN was slow and how Rtm fixed it
#17"It turns out there is a hack in FreeBSD, invented by Filo, that causes the O/S not to give a new connection to the server until an entire HTTP request has arrived." I wouldn't call it a hack, but a feature ;-) # Buffer a HTTP request in the kernel # until it's completely read. apache22_http_accept_enable="yes" Is HackerNews web scale?
What does "web scale" mean? I see it thrown around a lot without much explanation.
Re: Why HN was slow and how Rtm fixed it
#18HN only supports 20 req per second???
Maybe that's just for dynamic pages. Probably most of the most popular pages are cached and wouldn't be considered in that 20 req/sec. Just my own wild guess though.
Re: Why HN was slow and how Rtm fixed it
#19Reverse-proxying via nginx would solve this problem and more: the arbitrary 30 second limit on form submission (hotspots sometimes are slow...), nginx could handle rate limiting & logging instead of srv.arc, etc. The Arc codebase would btw be smaller and cleaner (no policy/sanitization code, etc.).
Serving static content via Apache was a first step ;-)
Don't reinvent the wheel!
Re: Why HN was slow and how Rtm fixed it
#20"It turns out there is a hack in FreeBSD, invented by Filo, that causes the O/S not to give a new connection to the server until an entire HTTP request has arrived." I wouldn't call it a hack, but a feature ;-) # Buffer a HTTP request in the kernel # until it's completely read. apache22_http_accept_enable="yes" Is HackerNews web scale?
What does "web scale" mean? I see it thrown around a lot without much explanation.