Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

App Engine down

code.google.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: App Engine down

#41

Post by tedroden »

davidjgraph wrote: post #6

Before the doom and gloomers come out, this is the first time since leaving beta I can remember it happening. We left AWS about 18 months ago after one of the outages and switched to GAE. I've counted 3-4 big downtimes for AWS compared to this one on GAE. That's still a good decision (for now)....

It's one of the first times the whole service has been down, but parts of the service go down at least once a week. memcache, task queues are "elevated" with regularity and urlfetch is frequently down totally. ("elevated" generally means unusable).

Of course master/slave even has scheduled downtime.

Re: App Engine down

#42

Post by Alan01252 »

tomgallard wrote: post #23
debacle wrote: post #12

I'm really happy I don't host in the cloud. How quickly are the cost savings of cloud computing obliterated by PR, customer service, and system administration time when an outage like this occurs?

Surely hosting yourself exposes you to just as much, if not more risk? Problem in the datacentre where you're co-lo'd, or one of your servers blows up? I think people not trusting the cloud is similar to how people feel safer driving their cars then taking a plane. The stats say the plane's safer, but people prefer being in control. People like the idea of being in control of their servers, even if that means there's…

I agree with you. From my past experience any data center is subject to risks. I've witnessed:

  Power failures.
  Cross site links being cut due to engineering works.
  Over heating due to air conditioning failures.
  Flooding
And I've experienced all the above from a very large, very well known, very expensive data center company based in London.

Re: App Engine down

#44

Post by abhijitr »

Meanwhile... Gmail etc are working quite fine. So the claim that if you build on GAE you "take advantage of the same infrastructure used for Google services!!" starts to ring a bit hollow.

Re: App Engine down

#46

Post by obituary_latte »

EwanToo wrote: post #40
davis_m wrote: post #17

I think this is larger than just GAE. http://internettrafficreport.com/namerica.htm It seems like large portions of the internet are down.

Internet Traffic Report, while a nice concept, is unfortunately very misleading. Their sample size is extremely small, and most of those are permanently down. Have a look through their list of north american routers and find one of them where packet loss has gotten worse as their main overall graph for packet loss would suggest - I've just been through them all and couldn't find one.

Agreed. I was surprised to see only one[1] of the "down" routers with any packet loss over the last 24 hours.

That said, it's still interesting how the overall traffic trends so sharply downwards. I wonder if they have more data than they are showing in the graphs.

[1]http://internettrafficreport.com/history/190.htm

Re: App Engine down

#47

Post by jre »

EwanToo wrote: post #40
davis_m wrote: post #17

I think this is larger than just GAE. http://internettrafficreport.com/namerica.htm It seems like large portions of the internet are down.

Internet Traffic Report, while a nice concept, is unfortunately very misleading. Their sample size is extremely small, and most of those are permanently down. Have a look through their list of north american routers and find one of them where packet loss has gotten worse as their main overall graph for packet loss would suggest - I've just been through them all and couldn't find one.

It seems like the individual router graphs most recent update is 10/26 00:00 whereas the overall graph has been updated at 10/26 08:30.

Re: App Engine down

#48

Post by peterwwillis »

tomgallard wrote: post #23
debacle wrote: post #12

I'm really happy I don't host in the cloud. How quickly are the cost savings of cloud computing obliterated by PR, customer service, and system administration time when an outage like this occurs?

Surely hosting yourself exposes you to just as much, if not more risk? Problem in the datacentre where you're co-lo'd, or one of your servers blows up? I think people not trusting the cloud is similar to how people feel safer driving their cars then taking a plane. The stats say the plane's safer, but people prefer being in control. People like the idea of being in control of their servers, even if that means there's…

The "safety" of the cloud is about two things: 1. trusting your service provider, and 2. redundancy.

You have to trust your cloud provider. They control everything you do. If their security isn't bulletproof, you're screwed. If their SAN's firmware isn't upgraded properly to deal with some performance issue, you're screwed. If their developers fuck up the API and you can't modify your instances, you're screwed. You have to put complete faith in a secret infrastructure run for hundreds of thousands of clients so there's no customer relationship to speak of.

That's just the "trust" issue. Then there's the issue of actual redundancy. It's completely possible to have a network-wide outage for a cloud provider. There will be no redundancy, because their entire system is built to be in unison; one change affects everything.

Running it yourself means you know how secure it is, how robust the procedures are, and you can build in real redundancy and real disaster recovery. Do people build themselves bulletproof services like this? Usually not. But if you cared to, you could.

Post reply on HN