171–180 of 202 posts
Lisp is definitely not a slow language: you can handle the crazy rate of 20 requests/second on a multi-core server!
If you choose to. We pushed north of 800 r/s in production, and just shy of 4k in our LAN, that's using stock hunchentoot with just mere customization . This guy here broke the 10k barrier: http://john.freml.in/teepeedee2-c10k
I worked at a startup once that made a network card that did this type of buffering (wait for whole HTTP requests, then forward as a lump to the host, across a local fast bus). Pretty whizzy, definitely helped server scaling. We started shipping in 2001; the dot-com bust more or less canceled any interest in the product, and canceled the company, too . . .
What was the name of the company/product?
Earlier quoted context omitted.
> and the rest of the page could be cached Except they can't, for the reasons I mentioned above. Eg, if my account is deaded, when I view a thread with one of my own comments, it looks different than if someone else was viewing that some thread, especially for those of us with or without showdead checked in our profiles. Its not as straightforward as you would like it to be.
The majority of requests probably come from live accounts in good standing or from people not even logged in, so the majority of requests could still be cached.
Earlier quoted context omitted.
Read C10K? Both select() and poll() have this problem internally. You have to use one of the more advanced techniques available if you really want to scale. epoll(), kqueue() or friends.
poll() is slightly better than select(), because you only have to iterate over the file descriptors that were passed, rather than from 0 to nfds.
Earlier quoted context omitted.
Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?
Pagination/"More" uses fnids; looking at the source it's a callback, but from an HTTP client perspective it might as well be a continuation. How do you test and debug things like that that have random URIs and function names and get GCed on a regular basis? That's what I mean when I say continuations lead to unmaintainable web apps.
Perhaps the reason I've never had a problem is that I've been careful to use this technique in fairly restricted ways. Lisp macros are the same sort of thing. They could yield unreadable code if abused. But good programmers use them in principled ways, and when used with restraint they are clearly a net win.
Since no one has mentioned it yet - Varnish-cache.org, written by a FreeBSD kernel hacker, has a very nice feature, in that it will put all overlapping concurrent requests for the same cacheable resource "on hold", only fetch that resource once from the backend, then serve the same copy to all. Nearly all the expensive content on HN would be cacheable by varnish. Then you can get it down to pretty close to "1 backend…
I feel silly asking, but who or what is 'rtm'?
Most recently, one of the creators of Arc (along with pg), the language used to create/run HN.
is there any reason the noobs url (29 Apr: Faster, Fewer Flamewars) ~ http://news.ycombinator.com/noobs shows no results?
Not being able to handle 20 requests/sec quickly, in 2011, for a read-mostly website is just shameful.
I think that all depends on the purpose of the site. If I were paying a subscription to access the site I would be within my rights to object. As it is, with HN being a free service, built to be the application spurring the development of Arc, whose reference implementation is intended for exploring language design and not performance, I wouldn't choose the word "shameful" to describe this situation.
Either way, it's 2011 and that really is some spectacular slowness.