Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku is down

heroku.com

Post reply on HN
Print view View original

61–70 of 107 posts

Re: Heroku is down

#61

Post by kitsune_ »

We've recently begun to move our entire infrastructure to external providers (vps and cloud) because we felt that we couldn't guarantee a sufficiently high level of quality and had to devote too much time and money to operations.

In over four months we've probably already had more downtime than in the past 5 years. Despite paying quite a lot for "redundant", clustered offerings. There was one nasty bug in the hypervisor that killed the entirety of one of our providers' vps infrastructure - for over a day.

Our experience with "the cloud" was a little bit better, but we aren't entirely satisfied.

I don't know, maybe we should just order services from three different providers mirror our applications ourselves as a fail-over mechanism.

Re: Heroku is down

#62

Post by ralphleon »

It's so scary to think that our entire startup almost died with this glitch. Our enterprise customers use our product in the mornings... this happened at 9-fucking-am right during our peak hours.

Obviously it's our fault for not having a redundant system / for trusting heroku. Though I still can't help but be a little pissed as I email 100 people about how sorry I am for their service disruption. Heroku didn't email me.

Re: Heroku is down

#63

Post by patio11 »

g0atbutt wrote: post #34

As a newbie, I've really enjoyed deploying on Heroku. However these outages really terrify me. Is there an easy way to host a redundant version of your app outside of Heroku?

Ask a simple question, get a simple answer: No. This is no knock on Heroku, either. No hosting solution can make that easy: it is an enterprise requirement which implies six figures of investment and a dedicated ops team with no newbies on it.

Re: Heroku is down

#64

Post by dholowiski »

kitsune_ wrote: post #61

We've recently begun to move our entire infrastructure to external providers (vps and cloud) because we felt that we couldn't guarantee a sufficiently high level of quality and had to devote too much time and money to operations. In over four months we've probably already had more downtime than in the past 5 years. Despite paying quite a lot for "redundant", clustered offerings. There was one nasty bug in the hypervi…

> maybe we should just order services from three different providers mirror our applications ourselves as a fail-over mechanism.

Yes, yes a thousand times yes. And dedicated servers don't cost much more than VPS's.

Re: Heroku is down

#65

Post by [deleted] »

wheels wrote: post #48
kennystone wrote: post #37

Earlier quoted context omitted.

Heroku is really good. You will probably have more downtime than them unless you hire a whole team of operations and admins and pay a lot of money for hosting. It may also take a while to build out the deployment and development tools they have.

Honestly, that's pious bullshit. It's the thing people keep saying every time Heroku gets mentioned is that it'd do better than you would yourself for availability, but you'd have to be borderline incompetent to have as much downtime with a more traditional VPS / hardware hosting to match the combined Heroku / AWS downtime. We moved all but one of our Rails apps off of Heroku precisely because of the frequent downtim…

[deleted]

Re: Heroku is down

#66

Post by patrickgzill »

believeUme wrote: post #45

Goes to show that the five nines is a bunch of crap. More like two nines.

5 nines is 5 minutes of downtime per year. 4 nines is 52 minutes of downtime per year.

See: http://en.wikipedia.org/wiki/High_availability#Percentage_ca... for a handy chart and follow along with any of your favorite *aaS providers!

Re: Heroku is down

#67

Post by wheels »

ricardobeat wrote: post #55
wheels wrote: post #48

Earlier quoted context omitted.

Honestly, that's pious bullshit. It's the thing people keep saying every time Heroku gets mentioned is that it'd do better than you would yourself for availability, but you'd have to be borderline incompetent to have as much downtime with a more traditional VPS / hardware hosting to match the combined Heroku / AWS downtime. We moved all but one of our Rails apps off of Heroku precisely because of the frequent downtim…

They are at 99.97% uptime. It's not that easy to get to even 99.5%, just the flakiness of a standard uplink will put you below that.

Huh? The flakiness of a standard uplink will have you down for 44 hours a year? On what planet? With our non-AWS hosting providers we tend to see 2-4 hours of network issues per year. Heroku's uptime also hasn't historically been anywhere near 99.97%. They were down for several days last year in The Great AWS Failure.

Re: Heroku is down

#68

Post by yourcelf »

lemieux wrote: post #59
bretthoerner wrote: post #40

http://www.whoownsmyavailability.com/

What is it suppose to do?

I think it's similar to http://isitchristmas.com/ . It's a simple answer that's always the same, to drive the point home: you are always the one responsible for your uptime, no matter whether you choose dedicated hosting, the cloud, your own closet, etc. You can't outsource responsibility.

Re: Heroku is down

#69

Post by patrickgzill »

kitsune_ wrote: post #61

We've recently begun to move our entire infrastructure to external providers (vps and cloud) because we felt that we couldn't guarantee a sufficiently high level of quality and had to devote too much time and money to operations. In over four months we've probably already had more downtime than in the past 5 years. Despite paying quite a lot for "redundant", clustered offerings. There was one nasty bug in the hypervi…

Yes. I recommend having a look at either ProxMox (proxmox.com) which can simultaneously do OpenVZ and KVM; or, XCP for Xen-based stuff - see http://www.xen.org/products/cloudxen.html

Re: Heroku is down

#70

Post by neilmiddleton »

kitsune_ wrote: post #61

We've recently begun to move our entire infrastructure to external providers (vps and cloud) because we felt that we couldn't guarantee a sufficiently high level of quality and had to devote too much time and money to operations. In over four months we've probably already had more downtime than in the past 5 years. Despite paying quite a lot for "redundant", clustered offerings. There was one nasty bug in the hypervi…

Heroku status is currently showing production uptime as 99.97%. That's not too shabby, and definitely not as bad as you make out.
Post reply on HN