Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Post mortem of a failed HackerNews launch

gigpeppers.com

Post reply on HN
Print view View original

41–50 of 107 posts

Re: Post mortem of a failed HackerNews launch

#41

Post by chacham15 »

politician wrote: post #40
mekoka wrote: post #36

Thanks for this post, there were some nice tips in there. Although, I do have some nitpicking about your writing style. Maybe it's just me, but I found that your use of "+ve" instead of just saying "positive" and of "&" instead of "and" did not have the intended effect of speeding up reading, quite the reverse actually.

Seconded. Initially, my brain told me that +ve was the name of the site, so I was confused when I looked for a product page link only saw "Cucumbertown". Granted, that's mostly laziness -- apparently I've got a rule that matches "strange words near the top of the post" to "probably the name of the product".

I dont know about anyone else, but when I saw "+ve", I just thought to myself, "what is that?" for about a half a second before giving up and moving on.

Re: Post mortem of a failed HackerNews launch

#42

Post by [deleted] »

chacham15 wrote: post #41
politician wrote: post #40

Earlier quoted context omitted.

Seconded. Initially, my brain told me that +ve was the name of the site, so I was confused when I looked for a product page link only saw "Cucumbertown". Granted, that's mostly laziness -- apparently I've got a rule that matches "strange words near the top of the post" to "probably the name of the product".

I dont know about anyone else, but when I saw "+ve", I just thought to myself, "what is that?" for about a half a second before giving up and moving on.

[deleted]

Re: Post mortem of a failed HackerNews launch

#44

Post by Cherian »

mekoka wrote: post #36

Thanks for this post, there were some nice tips in there. Although, I do have some nitpicking about your writing style. Maybe it's just me, but I found that your use of "+ve" instead of just saying "positive" and of "&" instead of "and" did not have the intended effect of speeding up reading, quite the reverse actually.

Thanks for the feedback. Corrected.

Call this a hacker’s laziness + Yahoo chat room era slangs.

Re: Post mortem of a failed HackerNews launch

#45

Post by haberman »

richardwhiuk wrote: post #37
lmm wrote: post #4

Running with swap enabled is a terrible idea. The authors mention how it was only once solr crashed that they were able to actually log in and start fixing problems; having swap means that rather than the OOM killer terminating processes, instead your whole system just grinds to a halt. (it's strange that they recommend enabling swap when they also recommend enabling reboot-on-oom, which is pretty much the complete o…

In my experience, the linux kernel handles no swap at all very badly, so you need a small amount. Increasing the swap, which is the suggested solution, is however, a terrible idea. As soon as you hit high memory usage, your IO load will go through the roof, and everything will grind to a halt. The solution here is separation of services - i.e. put Solr on a different box, so that if it spirals it doesn't take out oth…

> In my experience, the linux kernel handles no swap at all very badly, so you need a small amount.

Why?

I'm pretty sure we disable swap at Google. Maybe swap was necessary back in the days when memory was really tight, but it seems like a terrible idea now. Especially since the scheduling is completely oblivious to swap AFAIK, which means that a heavily swapped system will spend most of its timeslices just swapping program code back into memory. It's the worst kind of thrashing.

Re: Post mortem of a failed HackerNews launch

#46

Post by Cherian »

richardwhiuk wrote: post #37
lmm wrote: post #4

Running with swap enabled is a terrible idea. The authors mention how it was only once solr crashed that they were able to actually log in and start fixing problems; having swap means that rather than the OOM killer terminating processes, instead your whole system just grinds to a halt. (it's strange that they recommend enabling swap when they also recommend enabling reboot-on-oom, which is pretty much the complete o…

In my experience, the linux kernel handles no swap at all very badly, so you need a small amount. Increasing the swap, which is the suggested solution, is however, a terrible idea. As soon as you hit high memory usage, your IO load will go through the roof, and everything will grind to a halt. The solution here is separation of services - i.e. put Solr on a different box, so that if it spirals it doesn't take out oth…

You are right. The best solution is separation of services. But for a startup than runs 7-2 services like this – it’s a close call. You’ll often have to run 2-3 services together, else $100 * 7 machines is too much burn

Re: Post mortem of a failed HackerNews launch

#47

Post by ck2 »

1. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection).

2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to

3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddos. If you are reverse proxying to apache, connlimit is a MUST.

Re: Post mortem of a failed HackerNews launch

#48

Post by Cherian »

ck2 wrote: post #47

1. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection). 2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to 3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddo…

This is excellent advice! Thank you.

Re: Post mortem of a failed HackerNews launch

#50

Post by kyrra »

ChrisNorstrom wrote: post #25
buro9 wrote: post #5

If anyone owns a blog or site that they suspect may appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site. Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently. You should expect over 10,000 unique visitors within 24 hours, with most coming in the…

Oh, Well then let me share this. A few weeks ago I got on the front page and within a 24 hour period was hit with 29,000 unique visitors with 38,000 page views. The page itself is image heavy with 1.3 MB on first load. I'm running Wordpress with the Quick Cache plugin by PriMoThemes. I'm hosted on a shared 1and1 server. I've been hit before and went down, that's when I installed the Quick Cache plugin. Also 1and1 mov…

Agreed. I self-hosted a wordpress blog for a video-game fan site. I had a server status page that was a normal wordpress page that imported a file from the filesystem to embed in the middle (reported status of the game servers).

During some large events, I was seeing 5-7 page views per second and Wordpress did just fine (I think I had 120k page views in 24 hours). But I made sure to test as much of my site as possible to make sure it performed well. Using various page analyzers to make sure the proper headers were returned so objects were cached on the client. Tuning wordpress to cache pages properly. Turning on PHP APC (opcode cache). Running various stress tests on the site (apache benchmark and loadimpact).

I'd say the page analysis tools and extended apache-benchmark runs really helped me tune my OS and services properly so I could handle a huge load.

Post reply on HN