Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Heroku is Down (update: recovering as of 10PM PST)

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 116 posts

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#51

Post by hboon »

andrewcooke wrote: post #46
andrewvc wrote: post #28

Earlier quoted context omitted.

That's 99.97 for the last month . Which is ridiculous, 9s for most services use years as standard. Of course if heroku did that they wouldn't look so good.

if you have a year of 99.97 months, don't you have a 99.97 year? how does having a shorter time make the numbers look better? [edit: i realise they could be hiding worse months, but i don't think that's what the post i am replying to meant. perhaps i am reading it wrong.]

They could have terrible uptime 2 months ago and you wouldn't see it in their status page, because it only showed last month's.

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#52

Post by mparlane »

andrewcooke wrote: post #46
andrewvc wrote: post #28

Earlier quoted context omitted.

That's 99.97 for the last month . Which is ridiculous, 9s for most services use years as standard. Of course if heroku did that they wouldn't look so good.

if you have a year of 99.97 months, don't you have a 99.97 year? how does having a shorter time make the numbers look better? [edit: i realise they could be hiding worse months, but i don't think that's what the post i am replying to meant. perhaps i am reading it wrong.]

Depends on the month...

31 versus 28 days.

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#53

Post by ajasmin »

michaelfairley wrote: post #2

The AWS status page[1] is showing problems for EC2 East as of a few minutes ago. This might be is a more widespread issue. EDIT: Various non-Heroku EC2-East-based sites (e.g. Quora) seem to be down as well, lending more evidence to this being an EC2/EBS outage. 1: http://status.aws.amazon.com/

Out of curiosity do we know if http://status.aws.amazon.com/ is hosted on AWS?

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#54

Post by jasongullickson »

jasongullickson wrote: post #24

Seems like someone posted a link to a project that made self-hosting a previously Heroku hosted site simple, but I can't find it now... ...would be cool if there was a Linux package (or distro) that you could boot-up and then just change your git remote to and have your app up-and-running on your own hardware.

This is what I was thinking of (I think):

http://cloudfoundry.org/

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#55

Post by bcjordan »

zeeg wrote: post #47

Everyone should take a page from the book of Netflix right now. It's pretty embarrassing to be anyone that's entirely down and can't do a thing about it due to an EC2 outage. How do you explain to your customers/users/etc that you were down and have absolutely no control of when you will be back online? How can you explain it to yourself?

Netflix's "Lessons Learned" from the April 2011 AWS outage: http://techblog.netflix.com/2011/04/lessons-netflix-learned-...

Though they're not 100% degradation proof today either: http://i.imgur.com/MJfqj.png

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#57

Post by veemjeem »

andrewcooke wrote: post #46
andrewvc wrote: post #28

Earlier quoted context omitted.

That's 99.97 for the last month . Which is ridiculous, 9s for most services use years as standard. Of course if heroku did that they wouldn't look so good.

if you have a year of 99.97 months, don't you have a 99.97 year? how does having a shorter time make the numbers look better? [edit: i realise they could be hiding worse months, but i don't think that's what the post i am replying to meant. perhaps i am reading it wrong.]

They could have done it to the extreme -- show the numbers for the past hour. Then they could almost always report 100% uptime. And if they ever went down, wait an hour, then go back to reporting 100% uptime again.

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#58

Post by zred »

tg3 wrote: post #5

Their uptime percentage is 99.97%, but I'm having a hard time fighting the recency effect that is telling me to get off the platform ASAP.

That was their uptime for May, but looking at June it's going to be a worse picture. They're already down to 99.63% if you only include "red" incidents and down to 99.25% if you include "amber" incidents as well (as of 39m of downtime for this latest incident and assuming they don't have any more downtime for the month).

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#59

Post by ceejayoz »

ajasmin wrote: post #53
michaelfairley wrote: post #2

The AWS status page[1] is showing problems for EC2 East as of a few minutes ago. This might be is a more widespread issue. EDIT: Various non-Heroku EC2-East-based sites (e.g. Quora) seem to be down as well, lending more evidence to this being an EC2/EBS outage. 1: http://status.aws.amazon.com/

Out of curiosity do we know if http://status.aws.amazon.com/ is hosted on AWS?

status.aws.amazon.com resolves to 207.171.185.167 for me.

That IP isn't in the EC2 IP ranges (https://forums.aws.amazon.com/ann.jspa?annID=1528), so at the very least it's not hosted on EC2.

Re: Tell HN: Heroku is Down (update: recovering as of 10PM PST)

#60

Post by harryh »

zeeg wrote: post #47

Everyone should take a page from the book of Netflix right now. It's pretty embarrassing to be anyone that's entirely down and can't do a thing about it due to an EC2 outage. How do you explain to your customers/users/etc that you were down and have absolutely no control of when you will be back online? How can you explain it to yourself?

Because this shit is really hard especially when you're trying to build a product at the same time.

It's not like you can just wake up one day and say "I'm gonna go build a fully fault tolerant distributed system that works across multiple data centers!" and then you're done by the time you go to sleep.

Go actually talk to some Netflix engineers. They'll tell you the same thing.

Post reply on HN