Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why HN was slow and how Rtm fixed it

ycombinator.com

Post reply on HN
Print view View original

131–140 of 202 posts

Re: Why HN was slow and how Rtm fixed it

#131

Post by alain94040 »

I have a proposal to settle flamewars by the way. I had meant to propose something like this (a debate solver) for years. Here it is:

After 4 levels of back and forth (Joe says "...", Tim replies, then Joe replies once more, then Tim replies again), freeze that branch, hide it from the general public, and turn the branch into a settlement: both Tim and Joe are allowed one final comment each, that they both approve. Only once they have posted this compromise, is it shown in-place, where the original sub-thread used to be.

Simple. Prevents endless arguments. Good for everyone.

Re: Why HN was slow and how Rtm fixed it

#132

Post by axod »

pg wrote: post #129
axod wrote: post #127

Earlier quoted context omitted.

> If you read the source of HN, you'll see that it doesn't actually use continuations. > It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. (From further down the page). I'm confused. What needs continuations?

I just wanted to have them in the language. The fact that I don't currently use them in HN doesn't mean they're useless.

ah ok thanks for clarifying.

Re: Why HN was slow and how Rtm fixed it

#133

Post by marcamillion »

pg wrote: post #77
axod wrote: post #66

1 thread per connection??? Not doing continual GC in a separate thread and instead taking 7 seconds and blocking everything? What is this the 1990s?

Feel free to fork MzScheme and replace the garbage collector with a new one that runs continuously.

I love this response. Would be really interested in a follow-up from axod (or by pg about axod's version).

It's easy to criticize, but let's see what happens when the pedal hits the metal.

Re: Why HN was slow and how Rtm fixed it

#134

Post by marcamillion »

mbubb wrote: post #100

I find it disturbing to see people asking "Who is Rtm?" "Who is filo?" I understand if you are in tech you might not know figures in history or literature... but these guys? Every time you login to a UNIX/Linux system you use the passwd file and related setup - authored at least in part by Rtm's father. http://www.manpages.info/freebsd/passwd.1.html Rtm has done lots in his own right as the wikipdia pages show. But s…

This has to be one of the most unnecessary comments I have seen recently on HN.

Civility people. What happened to that?

Btw, I down vote me if you like, but it's true. It's easy for us to get caught up in our own brilliance that we talk down to others that don't know as much in a particular subject as we do.

Ironically, it shows more about you, than it does them.

Re: Why HN was slow and how Rtm fixed it

#135

Post by dauphin »

axod wrote: post #130
dauphin wrote: post #128

Earlier quoted context omitted.

Errors/exceptions, for one, are implemented using continuations.

Sounds terribly inefficient to me, but what do I know -shrug-

You know, if it weren't really impratical, numbers in Arc would be represented by Church numerals, "hash tables" would be alists, strings would be lists, etc.

That's a credo :-)

Re: Why HN was slow and how Rtm fixed it

#136

Post by aonic »

nuclear_eclipse wrote: post #106
aonic wrote: post #91

Earlier quoted context omitted.

Varnish supports edge side includes. The header bar could be an ESI and the rest of the page could be cached

> and the rest of the page could be cached Except they can't, for the reasons I mentioned above. Eg, if my account is deaded, when I view a thread with one of my own comments, it looks different than if someone else was viewing that some thread, especially for those of us with or without showdead checked in our profiles. Its not as straightforward as you would like it to be.

Special cookies could be set for dead users and users who enable showdead to bypass the cache.

For example, one of the sites I run has about 50K pageviews/day by logged in users, and another 600K pageviews/day by anonymous users coming from referrals or search engines. Logged in users have similar customization options so we bypass cache for these users by detecting a cookie.

Obviously going the cache route would require some changes to how things are setup, its not a turn-key solution. But the insignificant amount of changes are well worth it for most content sites, but for a user generated content site like HN it would also depend on how the TTLs and cache purging are setup.

Re: Why HN was slow and how Rtm fixed it

#137

Post by j_baker »

danudey wrote: post #122
j_baker wrote: post #121

Earlier quoted context omitted.

Doesn't that kind of defeat the purpose though? The point of using varnish is that it keeps you from having to access the backend altogether. This is getting into an area where something like memcache might be more appropriate.

Well, the point of using varnish is to keep you from having to access the backend any more than is absolutely necessary. It's incredibly trivial to generate HTML showing a user's username and karma, and even if it weren't it could be stored in memcached. Generating the front page, the comments pages, etc. is the hard part, and varnish can keep that from being generated any more than is necessary.

Of course, but I seem to recall pg writing at some point that one of the goals of HN being to prove that "slow" languages can scale using caching. I assume, therefore, that he already has caching of some kind in place for those things. If varnish isn't going to save an access to the server (which seems to be the primary thing that's slowing things down), what value is varnish providing above what pg already has in place?

Re: Why HN was slow and how Rtm fixed it

#138

Post by danudey »

j_baker wrote: post #137
danudey wrote: post #122

Earlier quoted context omitted.

Well, the point of using varnish is to keep you from having to access the backend any more than is absolutely necessary. It's incredibly trivial to generate HTML showing a user's username and karma, and even if it weren't it could be stored in memcached. Generating the front page, the comments pages, etc. is the hard part, and varnish can keep that from being generated any more than is necessary.

Of course, but I seem to recall pg writing at some point that one of the goals of HN being to prove that "slow" languages can scale using caching. I assume, therefore, that he already has caching of some kind in place for those things. If varnish isn't going to save an access to the server (which seems to be the primary thing that's slowing things down), what value is varnish providing above what pg already has in pl…

The requests won't queue up as badly because the server will be able to clean out 'simple' requests in a much lower time than generating much larger pages. They won't queue up as much because the requests take less time to handle, so they can be cleared out faster than they come in (compared to larger requests that queue up faster than they can be handled).

Re: Why HN was slow and how Rtm fixed it

#140

Post by shortlived »

axod wrote: post #93
pg wrote: post #77

Earlier quoted context omitted.

Feel free to fork MzScheme and replace the garbage collector with a new one that runs continuously.

Surely there's a way to target the JVM? I'd trust my life on the JVM, it's pretty battle tested, and the GC is simply awesome.

The JVM really is great and the garbage collecting outstanding. Anyone who has written a real high performance app can tell you that.
Post reply on HN