100% uptime for a web application • Hacker News Board

Re: 100% uptime for a web application

#11

Post by 300bps » Tue, May 28, 2013, 12:42 PM UTC

biot wrote: post #9

This is a great lesson in how not to be a pedantic nerd toward clients. By 100%, the client is obviously not specifying an uptime requirement of: lim (1 - 1/x) x->inf They're saying that downtime is unacceptable. As the consultant/contractor, it's your job to present various scenarios and reiterate to the client that redundancy/failover features introduced at each new level roughly doubles the hosting cost plus requi…

>This is a great lesson in how not to be a pedantic nerd toward clients

I've been in IT for 20 years so I may be jaded, but I disagree with your conclusion.

Your Levels don't seem to scratch the surface on achieving even 99.999% uptime. First of all, if you are planning on using vendors for your hypothetical 99.999% uptime you can stop there. Sure, they may promise that to you. But what is the penalty if they don't?

If you are planning on hosting this in your own data centers... Are you including diesel generators and massive diesel tanks to supply them in the event of extended power outages? What is your DDoS plan? What is your plan in the event of a hacker? What's your patching plan? These are just examples. The level of planning to achieve 99.999% uptime per year is massive. That's only 5 minutes of downtime per year!!

The bottom line is there are a handful of applications in the world that require this level of uptime. Unfortunately, every single client I have ever had was under the impression they required this level of uptime. It is my job to explain to them what reality is. If I don't, then I am not allowing them to prepare for the inevitable times that the system will be down no matter how much they spend or what they do.

Re: 100% uptime for a web application

#12

Post by biot » Tue, May 28, 2013, 12:56 PM UTC

300bps wrote: post #11

biot wrote: post #9

This is a great lesson in how not to be a pedantic nerd toward clients. By 100%, the client is obviously not specifying an uptime requirement of: lim (1 - 1/x) x->inf They're saying that downtime is unacceptable. As the consultant/contractor, it's your job to present various scenarios and reiterate to the client that redundancy/failover features introduced at each new level roughly doubles the hosting cost plus requi…

>This is a great lesson in how not to be a pedantic nerd toward clients I've been in IT for 20 years so I may be jaded, but I disagree with your conclusion. Your Levels don't seem to scratch the surface on achieving even 99.999% uptime. First of all, if you are planning on using vendors for your hypothetical 99.999% uptime you can stop there. Sure, they may promise that to you. But what is the penalty if they don't?…

I totally agree. I was assuming that diesel backup generators with refuelling contracts in place is considered table stakes for a data center these days. My example levels weren't exhaustive by any means and your points around capacity planning, attack mitigation, change management, etc. are spot on.

Re: 100% uptime for a web application

#13

Post by twistedpair » Tue, May 28, 2013, 4:25 PM UTC

300bps wrote: post #11

biot wrote: post #9

This is a great lesson in how not to be a pedantic nerd toward clients. By 100%, the client is obviously not specifying an uptime requirement of: lim (1 - 1/x) x->inf They're saying that downtime is unacceptable. As the consultant/contractor, it's your job to present various scenarios and reiterate to the client that redundancy/failover features introduced at each new level roughly doubles the hosting cost plus requi…

>This is a great lesson in how not to be a pedantic nerd toward clients I've been in IT for 20 years so I may be jaded, but I disagree with your conclusion. Your Levels don't seem to scratch the surface on achieving even 99.999% uptime. First of all, if you are planning on using vendors for your hypothetical 99.999% uptime you can stop there. Sure, they may promise that to you. But what is the penalty if they don't?…

Don't forget to plan for when your datacenter explodes and the building is condemned.

http://www.datacenterknowledge.com/archives/2008/06/01/explo...

Re: 100% uptime for a web application

#14

Post by eatmyshorts » Tue, May 28, 2013, 6:37 PM UTC

This doesn't address the asymptotic impossibility of maintaining true 100% uptime, but...

I think you can maintain uptime even when the servers aren't available with something like Meteor (a single-page web app built with Javascript...doesn't have to be Meteor, but Meteor should be able to handle it). It's not easy, but with Meteor's events indicating availability of the server should allow you to cache pending changes (when the server is unavailable) and send cached changes when the server becomes available again.

Meteor gives you full control over a client-side session, such that as long as the user doesn't reload the page, the user can continue using the app. There are hooks to detect when the server reconnects, so you should be able to write some custom code that updates the server with any pending changes that occurred when the server is unavailable.

Reloading the page would kill the pending updates, so you'd need to have the users use some sort of kiosk-like browser that prevents reloading the page and navigating to other pages.

There are other issues to be considered (what if the pending changes are invalidated by another user's changes?) that are very challenging, but can be handled depending on the user's other requirements.