Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why HN was slow and how Rtm fixed it

ycombinator.com

Post reply on HN
Print view View original

141–150 of 202 posts

Re: Why HN was slow and how Rtm fixed it

#141

Post by jrockway »

axod wrote: post #130
dauphin wrote: post #128

Earlier quoted context omitted.

Errors/exceptions, for one, are implemented using continuations.

Sounds terribly inefficient to me, but what do I know -shrug-

All control flow is a subset of continuations. The stack is a continuation (calling a function is call-with-current-continuation, return is just calling the "current continuation"), loops are continuations (with the non-local control flow, like break/last/redo/etc.), exceptions are continuations (like functions, but returning to the frame with the error handler), etc. Continuations are the general solution to things that are normally treated as different. So continuations are just as efficient (or inefficient) as calling functions or throwing exceptions.

In a web app context, though, it's kind of silly to keep a stack around to handler something like clicking a link that returns the contents of database row foo. People do this, call it continuations, and then run into problems. The problem is not continuations, the problem is that you are treating HTTP as a session, not as a series of request/responses. (The opposite of this style is REST.)

Re: Why HN was slow and how Rtm fixed it

#142

Post by jrockway »

swannodette wrote: post #62
jrockway wrote: post #46

Earlier quoted context omitted.

I don't know much about MzScheme, but it's quite possible that "thread" means "stack", not "OS thread". One context stack per TCP connection is quite sustainable; with Haskell's threads and Perl's coros, I run out of fds long before I'm using any significant amount of memory. (This is somewhere around 30,000 open connections on my un-tweaked Linux desktop. I know I can do a lot more if I tried.) The issue, in the cas…

My understanding is that MzScheme / Racket has a proper event loop.

Yeah, I have no idea. All I know is that proper threads do not bloat anything, and that proper IO watchers are not O(n) over inactive connections.

Re: Why HN was slow and how Rtm fixed it

#144

Post by dauphin »

taylorbuley wrote: post #101
mbubb wrote: post #100

I find it disturbing to see people asking "Who is Rtm?" "Who is filo?" I understand if you are in tech you might not know figures in history or literature... but these guys? Every time you login to a UNIX/Linux system you use the passwd file and related setup - authored at least in part by Rtm's father. http://www.manpages.info/freebsd/passwd.1.html Rtm has done lots in his own right as the wikipdia pages show. But s…

You're making little sense. So "go learn" but "don't ask?"

You can't ask a chalkboard to be coherent.

Re: Why HN was slow and how Rtm fixed it

#145

Post by jlouis »

axod wrote: post #66

1 thread per connection??? Not doing continual GC in a separate thread and instead taking 7 seconds and blocking everything? What is this the 1990s?

Let me add that one "thread" per connection works in Erlang for up to 80k connections easily. And since each "thread" is actually a process and each with their own GC, long pause-times are never a problem.

It is not the year, but the naivety that is the problem here, if any.

Re: Why HN was slow and how Rtm fixed it

#146

Post by sedachv »

pg wrote: post #126
sedachv wrote: post #115

Earlier quoted context omitted.

Or I don't know, use continuations in a place that's actually appropriate? John Fremlin showed that even with horrible CPS rewriting and epoll you can get way better throughput in SBCL (TPD2) than nginx. MzScheme comes with native continuations. It's not hard to call out to epoll. Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable we…

Instead everyone in the Lisp community (pg included) is still enamored with using continuations to produce ugly URLs and unmaintainable web applications. If you read the source of HN, you'll see that it doesn't actually use continuations. I find the source of HN very clear. Have you read it? Is there a specific part you found so complicated as to be unmaintainable?

Pagination/"More" uses fnids; looking at the source it's a callback, but from an HTTP client perspective it might as well be a continuation.

How do you test and debug things like that that have random URIs and function names and get GCed on a regular basis? That's what I mean when I say continuations lead to unmaintainable web apps.

Re: Why HN was slow and how Rtm fixed it

#147

Post by brc »

alain94040 wrote: post #131

I have a proposal to settle flamewars by the way. I had meant to propose something like this (a debate solver) for years. Here it is: After 4 levels of back and forth (Joe says "...", Tim replies, then Joe replies once more, then Tim replies again), freeze that branch, hide it from the general public, and turn the branch into a settlement: both Tim and Joe are allowed one final comment each, that they both approve. O…

Hmm, I have had ideas for something similar, but which involves having to choose a side of the argument (ie, agree with parent or disagree) before posting. Once chosen, you can only vote on your 'side' (either up or down). Poor arguments on your 'side' can be killed with sufficient downvotes, so that the ensuing set of arguments hopefully ends up being the best set. This tends to happen in an informal way on HN, but only because people largely behave. In other forums, not so much. Perhaps glomming together your idea of maximum posts per user on a topic, along with side-based voting, some type of civil debating platform coudl be developed. After all, in actual debates you get 2 chances to state your position and a final sum-up.

Re: Why HN was slow and how Rtm fixed it

#148

Post by ezalor »

jrockway wrote: post #141
axod wrote: post #130

Earlier quoted context omitted.

Sounds terribly inefficient to me, but what do I know -shrug-

All control flow is a subset of continuations. The stack is a continuation (calling a function is call-with-current-continuation, return is just calling the "current continuation"), loops are continuations (with the non-local control flow, like break/last/redo/etc.), exceptions are continuations (like functions, but returning to the frame with the error handler), etc. Continuations are the general solution to things…

Eat this: "The continuation that obeys only obvious stack semantics, O grasshopper, is not the true continuation." -- Guy Steele

Re: Why HN was slow and how Rtm fixed it

#149

Post by nitrogen »

nuclear_eclipse wrote: post #106
aonic wrote: post #91

Earlier quoted context omitted.

Varnish supports edge side includes. The header bar could be an ESI and the rest of the page could be cached

> and the rest of the page could be cached Except they can't, for the reasons I mentioned above. Eg, if my account is deaded, when I view a thread with one of my own comments, it looks different than if someone else was viewing that some thread, especially for those of us with or without showdead checked in our profiles. Its not as straightforward as you would like it to be.

The majority of requests probably come from live accounts in good standing or from people not even logged in, so the majority of requests could still be cached.

Re: Why HN was slow and how Rtm fixed it

#150

Post by ezalor »

pg wrote: post #129
axod wrote: post #127

Earlier quoted context omitted.

> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?

I just wanted to have them in the language. The fact that I don't currently use them in HN doesn't mean they're useless.

I always thought the purpose of Arc was to be cruft-free, "don't include it unless it is actually needed".
Post reply on HN