81–90 of 107 posts
1. HN should let you pay them $10 and let them hammer your server(s) before your story goes live. good for you. good for them.
2. there's a deal at lowendbox right now for a 2GB VPS for $30 a YEAR. you could have a healthy server farm for pretty cheap.
This post mortem has me thinking about the best way to handle the situation in which you can't SSH into your server. The OP decided to trigger a kernel panic/restart on OOM errors, but I have a couple of concerns about this approach: * If memory serves correctly, if your system runs out of memory, shouldn't the scheduler kill processes that are using too much memory? If this is the case, the system should recover fro…
* I've never seen any sort of virtual hosting service without either a remote console or a remote reboot. Usually both.
If anyone owns a blog or site that they suspect may appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site. Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently. You should expect over 10,000 unique visitors within 24 hours, with most coming in the…
Oh, Well then let me share this. A few weeks ago I got on the front page and within a 24 hour period was hit with 29,000 unique visitors with 38,000 page views. The page itself is image heavy with 1.3 MB on first load. I'm running Wordpress with the Quick Cache plugin by PriMoThemes. I'm hosted on a shared 1and1 server. I've been hit before and went down, that's when I installed the Quick Cache plugin. Also 1and1 mov…
AwStats screen shot: https://unavailable.s3.amazonaws.com/20121130_WP-Super-Cache...
If anyone owns a blog or site that they suspect may appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site. Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently. You should expect over 10,000 unique visitors within 24 hours, with most coming in the…
I'll be very happy to help you and HN'ers set Varnish up on their server (not looking for compensation for this) and get you through HN traffic on your launch day. Plug: we've built several products around Varnish so we have a good handle of how/where Varnish can be leveraged. Here's a list of varnish things we've built at unixy: Varnish load balancer: http://www.unixy.net/advanced-hosting/varnish-load-balancer Varni…
A few things have caught my attention in your post. Your biggest problem was that the configuration of your services was not sized/tuned properly for the hardware resources you've got. As a result of this your servers have become unresponsive and instead of fixing the problem, you've had to wait 30+ minutes until the servers recovered. In your case you should have limited Solr's JVM memory size to the amount of RAM t…
I mean, careful configuration and capacity planning is important. But what happened to straightforward conservative hardware purchasing where you get a much bigger system than you think you need? It's not like bigger hosts are that expensive: splurge for an EC2 2XL ($30/day I think) or three for the week you launch and have a simple plan in place for bringing up more to handle bursty load.
[1] The OOM killer picked a 2.7G Java process to kill. It usually picks the biggest thing available, so I'm guessing at 4G total.
Earlier quoted context omitted.
Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth.
> Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth. Bad idea. This way you're actively increasing the latency of your site. This way, for each asset that has to be fetched you're forcing the client to open a new connection, which can add more than 150 ms of delay per item (thanks to the three way TCP handshake). What I would sugg…
It's great for an ajax-heavy site, especially when it's all behind SSL. Using a CustomLog I log total request time, from connection time to the request has been served (conditional log when the request is handled by the backend) and I can see it's halved since I could use the threaded Apache.
Currently I have 100 threads per Apache process, and ~20 of them handling 2000 idle connections. I'm sure this can be tweaked some more.
Apaches manages SSL etc. and just proxies to my application servers.
There's also an event-based Apache module which I haven't tried.
Thanks for this post, there were some nice tips in there. Although, I do have some nitpicking about your writing style. Maybe it's just me, but I found that your use of "+ve" instead of just saying "positive" and of "&" instead of "and" did not have the intended effect of speeding up reading, quite the reverse actually.
Agree with the former, but it bears saying: you're about 2 millenia too late to be complaining about the use of the ampersand. :)
Ampersands are often used to mark brands, names and cultural items made up of multiple components: Johnson & Johnson, Dungeons & Dragons, bread & butter, fish & chips, Gold, Smith & Associates.
If I say "I had some fish & coleslaw" that would make a few people wonder if this is some popular recipe they should google.
Earlier quoted context omitted.
I'll be very happy to help you and HN'ers set Varnish up on their server (not looking for compensation for this) and get you through HN traffic on your launch day. Plug: we've built several products around Varnish so we have a good handle of how/where Varnish can be leveraged. Here's a list of varnish things we've built at unixy: Varnish load balancer: http://www.unixy.net/advanced-hosting/varnish-load-balancer Varni…
A how-to guide would be really really cool.
You could come up with a generic VCL that works for most websites out there but its cache effectiveness diminishes as you try to account for the most common corner cases. In fact, we did come up with such VCL. We distribute it with the cPanel Varnish plugin.
If you ever have a question or need a hand with Varnish/VCL drop me an email. I'll be more than happy to help out.
Earlier quoted context omitted.
Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth.
> Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth. Bad idea. This way you're actively increasing the latency of your site. This way, for each asset that has to be fetched you're forcing the client to open a new connection, which can add more than 150 ms of delay per item (thanks to the three way TCP handshake). What I would sugg…
There might be some magic value at which KeepAlive will be helpful during non-peak periods without crippling the server during peak periods, but for a well-engineered site, the extra 10ms delay per request shouldn't be a big enough deal to warrant risking a full-on site outage later on.
Also, this has already been discussed to death on HN: http://www.hnsearch.com/search#request/all&q=keepalive&#...
Earlier quoted context omitted.
Oh, Well then let me share this. A few weeks ago I got on the front page and within a 24 hour period was hit with 29,000 unique visitors with 38,000 page views. The page itself is image heavy with 1.3 MB on first load. I'm running Wordpress with the Quick Cache plugin by PriMoThemes. I'm hosted on a shared 1and1 server. I've been hit before and went down, that's when I installed the Quick Cache plugin. Also 1and1 mov…
Thank you. This is pretty much exactly what I read the post for and was a little disappointed not to find. People talk about getting frontpaged on HN, Slashdot or Reddit, and how you need to be sure you can handle the load, but never give any useful figures on what that load is. Knowing that I can serve 10 requests per second and likely withstand a frontpage on HN is more useful to me than knowing that I need a way o…
http://measuredmass.wordpress.com/2012/10/12/hacker-news/
Graphical post-mortem here:
http://measuredmass.wordpress.com/2012/10/20/more-hn-numbers...
