Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Client asks for 100% uptime

serverfault.com

Post reply on HN
Print view View original

51–60 of 100 posts

Re: Client asks for 100% uptime

#51

Post by duck »

msy wrote: post #2

It's always helpful when clients give unambiguous signs of unreasonable insanity upfront instead of hiding it until you're halfway through the project. It makes running away as far and fast as humanly possible so much easier.

There also was this comment on the SO thread: I would personally RUN from this client as fast as possible. I suspect this won't be the last crazy idea they may have (from a technology standpoint).

Why run though? They probably just don't understand what 100% means and it just takes you explaining it to them. Or simply state that you cannot meet that requirement and see if they still want you to bid on the project.

Re: Client asks for 100% uptime

#52

Post by davidcuddeback »

Retric wrote: post #48
oconnore wrote: post #31

Earlier quoted context omitted.

No, three 9 reliability for a single server. 1-.001^3 = .999999999, which is under a second expected per year, which the client will never notice even with good monitoring tools, and therefore will never invoke the contract.

Your assuming independence to a level that does not exist. Consider a Y2K style bug in the OS would could take down all severs for an extended period of time. Or someone could write a virus that uses a zero day exploit etc.

I think it's more likely that the programmer screws up in keeping the separate servers independent through database migrations.

Re: Client asks for 100% uptime

#53

Post by gte910h »

This is not a technical issue, this is a contractual issue. You want to sign a contract that pays reasonable amounts for reasonable (far less than 100%) reliability for every minute of downtime.

Then you have the impetus to minimize (but not eliminate) those minutes to a reasonable level.

Re: Client asks for 100% uptime

#54

Post by sunchild »

sunchild wrote: post #20

100% uptime is not an operational requirement – it's contractual. A client that demands 100% uptime isn't being unreasonable; they're looking for a contract remedy (most likely a termination right) if/when the site goes down. 1. "Uptime" is defined in many, many ways. In the OP's article, it's the definition of uptime that seems unreasonable. Normally, the demarc points for the network segments and equipment being me…

EDIT: I've represented a whole lot of customers of web services over the years (IAAL). The big lesson in this area for me is: (1) customers rarely invoke SLA credits, preferring instead to "work it out" at the relationship level, and (2) most provider off-the-shelf SLAs are so full of holes and tricky thresholds that they are effectively useless. On this last point, beware the 100% or five nines or other unreasonably high uptime commitment. When you get into the details of the SLA (the demarcs, the qualifications for obtaining credits, the remedies for failure), you will almost always find that there is no realistic remedy at all.

N.B. Meant to edit my comment above, not self-respond.

Re: Client asks for 100% uptime

#55

Post by Colman »

sophacles wrote: post #37
nodata wrote: post #34

99.5% uptime is 100% uptime to 0 decimal places.

I appreciate the snark in your post :), but it also brings up a serious question: How does contract law handle significant figures?

I haven't read any cases on this, but I imagine most courts would truncate/round down the achieved performance number. If the contract says 100%, that wouldn't allow for any downtime whatsoever.

It might be different for lower percentages - getting 89.8% performance where 90% is called for could be a de minimis breach and not actually count as breaking the contract. Definitely curious as to whether anyone has more to add on this.

Re: Client asks for 100% uptime

#56

Post by swombat »

snorkel wrote: post #42
oconnore wrote: post #26

I don't understand what the issue is. The client wants you to plan for disaster, and they aren't math oriented, so asking for 100% probability sounds reasonable. The engineer, as engineers are prone to do, remembered his first day of prob&stat 101, without considering that the client might not. When they say this, they aren't thinking about nuclear winter, they are thinking about Fred dumping his coffee on the office…

Most clients don't understand the R2D2 talk. They understand money, features, bugs, and downtime. So I've always explained it like so: Uptime beyond 95% costs lots and lots of money. Magnitudes of order of more money. It requires redundant equipment, engineering all of the automatic failovers at every layer, lots of monitoring, and 24/7 technical staff to watch everything like a hawk. Not ... in ... your ... budget.…

First of all, uptime of 95% means 18 and a quarter entire days of downtime per year. That's horrendous. I wouldn't host my dog's website on a server with that kind of SLA - and I don't even have a dog.

Secondly, although Twitter got away with large helpings of downtime, that doesn't mean that every business type can. Twitter is not (or at least was not, for most of its existence) business-critical to anyone. If Twitter goes down, oh well. Shucks.

If you're running recently-featured-on-HN Stripe, however, where thousands or more other businesses depend on you keeping your servers up to make their money, I'd say even 10 minutes of downtime is unacceptable.

Finally, this doesn't have to cost a lot. Just find a host that offers the SLA you're looking for, and have a reasonably fast failover to another similar host somewhere else.

Re: Client asks for 100% uptime

#58

Post by oconnore »

Retric wrote: post #48
oconnore wrote: post #31

Earlier quoted context omitted.

No, three 9 reliability for a single server. 1-.001^3 = .999999999, which is under a second expected per year, which the client will never notice even with good monitoring tools, and therefore will never invoke the contract.

Your assuming independence to a level that does not exist. Consider a Y2K style bug in the OS would could take down all severs for an extended period of time. Or someone could write a virus that uses a zero day exploit etc.

I am making no such assumption. See [1] in my original post. I already talked about intersection. Feel free to add Y3K to the list of nuclear war, chinese hackers, etc. The intersection is incredibly small, and not something that I am going to include in my back-of-the-napkin calculation.

Re: Client asks for 100% uptime

#59

Post by alexqgb »

brown9-2 wrote: post #28
oconnore wrote: post #26

I don't understand what the issue is. The client wants you to plan for disaster, and they aren't math oriented, so asking for 100% probability sounds reasonable. The engineer, as engineers are prone to do, remembered his first day of prob&stat 101, without considering that the client might not. When they say this, they aren't thinking about nuclear winter, they are thinking about Fred dumping his coffee on the office…

But "three 9 reliability" is still not the same thing as 100%. The contractor has a right to be concerned about the 100% figure making it's way into a contract.

Um, things don't just "make their way" into contracts. Yes, they suddenly appear in drafts, but finals? Sorry, no. Finals require approvals and signatures from the people who are going to be on the hook. (The e-Bay lawyers who approved the Skype purchase may beg to differ, but they're hardly unbiased.)

The draft-stage is where you take snorkel's exactly-right advice, and declare that the difference between 99.999% and 100% is about a bajillion dollars, give-or-take. Go a step further, and sell that 0.001% hard by pointing out just how much they're be prepared for ("multiple earthquakes plus a giant robot attack, all at once!"). They'll start rethinking fast - guaranteed.

And that's the essence of diplomacy; letting the other guy have it your way. If he thinks it's his idea, even better.

Re: Client asks for 100% uptime

#60

Post by shabda »

oconnore wrote: post #58
Retric wrote: post #48

Earlier quoted context omitted.

Your assuming independence to a level that does not exist. Consider a Y2K style bug in the OS would could take down all severs for an extended period of time. Or someone could write a virus that uses a zero day exploit etc.

I am making no such assumption. See [1] in my original post. I already talked about intersection. Feel free to add Y3K to the list of nuclear war, chinese hackers, etc. The intersection is incredibly small, and not something that I am going to include in my back-of-the-napkin calculation.

Your failover code never has bugs?

See the linked discussion, only 3 out of 20 top sites had 5 nines.

Post reply on HN