Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why HN was slow and how Rtm fixed it

ycombinator.com

Post reply on HN
Print view View original

121–130 of 202 posts

Re: Why HN was slow and how Rtm fixed it

#121

Post by j_baker »

seiji wrote: post #85
j_baker wrote: post #78

Earlier quoted context omitted.

Can't you only use varnish for mostly non-dynamic content? Like for example, wouldn't the fact that it displays my username and karma score at the top of the page make it so that you couldn't use varnish (or at least make it more difficult)?

Check out http://www.varnish-cache.org/trac/wiki/ESIfeatures There could be a private internal URL to just return username and karma to populate the user info header.

Doesn't that kind of defeat the purpose though? The point of using varnish is that it keeps you from having to access the backend altogether. This is getting into an area where something like memcache might be more appropriate.

Re: Why HN was slow and how Rtm fixed it

#122

Post by danudey »

j_baker wrote: post #121
seiji wrote: post #85

Earlier quoted context omitted.

Check out http://www.varnish-cache.org/trac/wiki/ESIfeatures There could be a private internal URL to just return username and karma to populate the user info header.

Doesn't that kind of defeat the purpose though? The point of using varnish is that it keeps you from having to access the backend altogether. This is getting into an area where something like memcache might be more appropriate.

Well, the point of using varnish is to keep you from having to access the backend any more than is absolutely necessary. It's incredibly trivial to generate HTML showing a user's username and karma, and even if it weren't it could be stored in memcached. Generating the front page, the comments pages, etc. is the hard part, and varnish can keep that from being generated any more than is necessary.

Re: Why HN was slow and how Rtm fixed it

#126

Post by pg »

sedachv wrote: post #115
mmaunder wrote: post #99

"In 7 seconds, a hundred or more connections accumulate. So the server ends up with hundreds of threads, most of them probably waiting for input (waiting for the HTTP request). MzScheme can be inefficient when there are 100s of threads waiting for input -- when it wants to find a thread to run, it asks the O/S kernel about each socket in turn to see if any input is ready, and that's a lot of asking per thread switch…

Or I don't know, use continuations in a place that's actually appropriate? John Fremlin showed that even with horrible CPS rewriting and epoll you can get way better throughput in SBCL (TPD2) than nginx. MzScheme comes with native continuations. It's not hard to call out to epoll. Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable we…

Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications.

If you read the source of HN, you'll see that it doesn't actually use continuations.

I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?

Re: Why HN was slow and how Rtm fixed it

#127

Post by axod »

pg wrote: post #126
sedachv wrote: post #115

Earlier quoted context omitted.

Or I don't know, use continuations in a place that's actually appropriate? John Fremlin showed that even with horrible CPS rewriting and epoll you can get way better throughput in SBCL (TPD2) than nginx. MzScheme comes with native continuations. It's not hard to call out to epoll. Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable we…

Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?

> If you read the source of HN, you'll see that it doesn't actually use continuations.

> It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best.

(From further down the page).

I'm confused. What needs continuations?

Re: Why HN was slow and how Rtm fixed it

#128

Post by dauphin »

axod wrote: post #127
pg wrote: post #126

Earlier quoted context omitted.

Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?

> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?

Errors/exceptions, for one, are implemented using continuations.

Re: Why HN was slow and how Rtm fixed it

#129

Post by pg »

axod wrote: post #127
pg wrote: post #126

Earlier quoted context omitted.

Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?

> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?

I just wanted to have them in the language. The fact that I don't currently use them in HN doesn't mean they're useless.

Re: Why HN was slow and how Rtm fixed it

#130

Post by axod »

dauphin wrote: post #128
axod wrote: post #127

Earlier quoted context omitted.

> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?

Errors/exceptions, for one, are implemented using continuations.

Sounds terribly inefficient to me, but what do I know -shrug-
Post reply on HN