Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cause of YC/HN outage discovered

archub.org

Post reply on HN
Print view View original

61–70 of 110 posts

Re: Cause of YC/HN outage discovered

#61

Post by Locke1689 »

axod wrote: post #12
pg wrote: post #4

Things should be back to normal now. Trevor just moved www.ycombinator.com to Slicehost, and I just told HN to refer to static stuff there instead of serving it locally. Interesting how easy it is to move your whole web site. Good customer service is important when users can switch so easily.

>> "Things should be back to normal now." Hopefully that's the normal of months ago when loading comments/threads etc was quick? Still taking 10-20 seconds to load most pages. One thing I haven't tried yet is just creating a fresh account - maybe mine just has too much associated with it (Maybe I comment too much etc). edit: ah I see it only affected static content. Shame :/

I've had some speed issues too. My account isn't tiny, but it is a 12th of your upvote count.

Re: Cause of YC/HN outage discovered

#62

Post by jseliger »

bwb wrote: post #27

Guys, shared hosting is designed for the 99% of non busy sites, say under 1,500 unique visitors a day, with the level of traffic HN is doing I'm surprised Pair didn't warn you earlier. A site this busy needs a dedicated server or virtual server. That is why shared hosting is cheap, you start with it and once you are successful or starting to get slashdotted you buy something bigger that can scale.

I don't think anyone will disagree. What they will assert, however, is that shutting down a site with zero notice is a jerky thing to do. As the other commenters have said, the nicer/smarter thing to do would've been to send an e-mail that says "Hey, you're using too much bandwidth/whatever." The smartest thing of all probably would've been, "You're using too much. Want to by more?"

Re: Cause of YC/HN outage discovered

#63

Post by ramchip »

jqueryin wrote: post #29

I actually find it quite embarrassing that the guy had no clue what HN or YC were. He had absolutely no idea... and he works for a hosting company. It's quite a shame. I really got a kick out of the fact he questioned the legitimacy of traffic as well.

I don't really see why every hosting company employee out there is supposed to know about a certain small american investment firm started in 2005 and its associated news aggregator... we're probably an order of magnitude or so less popular than Reddit, which is itself rather niche. It's a small dot in the web world.

Re: Cause of YC/HN outage discovered

#64

Post by zck »

gojomo wrote: post #54
neurotech1 wrote: post #53

Earlier quoted context omitted.

Yes and no. I doubt it is a HN-Focused tool, as this affected the static content from www.ycombinator.com, and images and CSS are not the focus of bots usually. Assuming it was not a DoS attack, a smart host should have noticed the traffic load increasing over time, and offered to upgrade to a less-loaded server and recommend a dedicated server. By disabling the site, they have lost a customer, and lost on a up-sell…

You're assuming the load today wasn't extraordinary. That's not yet clear. PG doesn't know of any surge, probably based on figures he has access to on other hosts and analytics tools. But what if HN/YC static resources at Pair were newly being deep-linked, by a much more popular site? Only seeing the end-of-day logs from Pair, compared to previous days, can definitively answer this question.

tlb says it was "gradual": http://news.ycombinator.com/item?id=1283439 .

Re: Cause of YC/HN outage discovered

#65

Post by rg »

bwb wrote: post #37
bwb wrote: post #27

Guys, shared hosting is designed for the 99% of non busy sites, say under 1,500 unique visitors a day, with the level of traffic HN is doing I'm surprised Pair didn't warn you earlier. A site this busy needs a dedicated server or virtual server. That is why shared hosting is cheap, you start with it and once you are successful or starting to get slashdotted you buy something bigger that can scale.

Also, just going to point out here that Pair.com has had a great reputation in the hosting industry, and has for the last 10 years. I've been in the industry for 10 years and worked for quite a number of hosting companies, not Pair though, and when you have 150 shared clients on a machine and 1 client is causing the problems you do your best to deliver the warning before it gets out of hand but it is very hard to do.

When I emailed Pair.com to tell them that my site was about to get a writeup in the Wall Street Journal, they moved my site to a dedicated server within an hour, left it there until the spike of traffic was over, then moved my site back to a shared server--all at no charge. If you use Pair.com properly, they are more than excellent.

Re: Cause of YC/HN outage discovered

#66

Post by nkassis »

archon810 wrote: post #6

This is almost as bad as the reason for my server's downtime recently: http://beerpla.net/wp-content/uploads/img_2958.JPG My server literally had its plug pulled.

Are we at the point were we have to deal with robotic janitors too?

Geez, and I thought that watching the janitors while the work was over the top, now I gotta watch the rumba too :(

Re: Cause of YC/HN outage discovered

#67

Post by gojomo »

zck wrote: post #64
gojomo wrote: post #54

Earlier quoted context omitted.

You're assuming the load today wasn't extraordinary. That's not yet clear. PG doesn't know of any surge, probably based on figures he has access to on other hosts and analytics tools. But what if HN/YC static resources at Pair were newly being deep-linked, by a much more popular site? Only seeing the end-of-day logs from Pair, compared to previous days, can definitively answer this question.

tlb says it was "gradual": http://news.ycombinator.com/item?id=1283439 .

I'm sure that's what the general News.YC/YC stats show. But as the Pair.com tech mentioned, today's logs only become available at midnight ET. If there was a surge this morning -- to the static resources only, as if by offsite deep-linking -- Pair's logs are the only place it would be evident.

Re: Cause of YC/HN outage discovered

#68

Post by [deleted] »

pavs wrote: post #57
CGamesPlay wrote: post #43

Earlier quoted context omitted.

VPSes are still shared hosting and are not immune from the problem.

They won't shut you down for over-using resources. As a matter of fact you can't over-use resources, you can only use the resources you are given to you. I know they are different type of VPS, I am referring to Slicehost/Linode type VPS which has hard limit and acts very much like dedicated single box. More important to me is that I am really really really surprised that Ycombinator was running on a shared account. T…

[deleted]

Re: Cause of YC/HN outage discovered

#69

Post by BrandonM »

gojomo wrote: post #28
sh1mmer wrote: post #22

Dear Pair Networks, I'd like to highlight this lesson in how to loose (or not gain) customers by randomly shutting down technology sites that serve the decision makers you wish to influence. Tom P.S. Well done Slicehost.

150K hits in 30 minutes is 83 hits/second. That's a lot to ask for from a shared-hosting account. When the traffic starts to impair neighboring sites, something has to be done. Just about any ISP will do the same thing: block the site with the surge, that could possibly make other arrangements, rather than inconvenience other customers whose traffic is as expected/usual. The detail missing so far is why Pair noticed…

So let's see... you're hosting a website that has gotten large; that is, it's grown to the point that it will need higher-cost services in order to meet demand. You have a chance to add a valuable customer to your client base. How best to handle this?

a) anything

b) except

c) killing their service

Re: Cause of YC/HN outage discovered

#70

Post by Xichekolas »

bwb wrote: post #47
eli wrote: post #41

Earlier quoted context omitted.

Anyone can get slashdotted. And my time is valuable -- I'd much rather pay more for better service.

Ya but if your dad wants to put up his pictures you go tell him to buy a nice scalable hosting product instead of a cheap shared hosting account for $5 to $10 a month?

No. I'd tell him to get a Facebook/Picasa/Flickr/Twitpic account.

Dad used to get a shared hosting account for things like that, but now we have services that host content for free. They are much easier for Dad to use as well.

In this age of $20/mo VPSs and free content hosting, I'm honestly not sure how shared hosting survives. It's not as flexible as a VPS for hosting sites (and with more than two sites, isn't even as cheap), and it's not as easy to use (or as free) as Facebook.

Post reply on HN