Live data from Hacker News

Why Are Projects Always Behind Schedule?

priceonomics.com

51–60 of 111 posts

Re: Why Are Projects Always Behind Schedule?

#51

Earlier quoted context omitted.

According to this reasoning you should set targets based on expected time, not "median" time. You're saying the tail of "completion time" is long, and so the median is a poor estimator for expected time. But if you actually knew the distribution of times for task completion (which is implied if you know the 'median time'), you could simply use the expected value of this distribution and your projects would then tend…

Exactly. All this article is really saying is "most of the time, adding together median expected times underestimates the sum of mean expected times". Well then, easy fix: add together the mean expected times off the bat.

Bingo. When we think about the steps intuitively, we tend to gravitate toward the median. "How long does this usually take?" is asking about a median in surprisingly precise terms.

Instead we have to understand the distribution, and the mean as well, and add up how long it takes on average, rather than just in the most numerous case.

Further, and perhaps more importantly, by understanding each distribution, you have far greater knowledge about what the expected time will be, and where the delays are most probable; and you can begin to start analyzing where those delays might be coming from in the interactions of all the parts of the system. This is straight-up quality talk right out of W. Edwards Deming, and it's how you start to improve how you produce in general. Good stuff.

Re: Why Are Projects Always Behind Schedule?

#52
post #37

The fact that planning estimates represent a probability distribution is well known and there is an established process (PERT)[1] for estimating the expected time of a set of estimates - essentially instead of asking for a single 'most likely' estimate you should as well explicitly walk through some of the risks around this and the ask for an 'optimistic' estimate (which is frequently very similar to the initial esti…

The typical PERT three point distribution is better than assuming the mean, but does have a few drawbacks when modeling joint probable outcomes. I honestly can't remember if it over our under represents the optimistic or pessimistic result.

The other quirk is that it breaks down when you map from effort or work to duration, the actual time something takes when you factor in availability, interruptions, communication costs, etc.

Re: Why Are Projects Always Behind Schedule?

#53

Time estimations is an industrial way of thinking applied to a post-industrial world. In the post industrial world time isn't the problem but rather project definition and scoping. In the industrial world the problem was already solved (machine was built, market often established and output depended on a few factors that could be adjusted. Need more output add more of X) In the post industrial world every project is…

That's a good point, but it has two issues you should consider.

First, the industrial world also had significant issues with project definition and understanding. The problems were not already solved, and the process, inputs, outputs and the whole system was constantly changing. Every project was already about problem solving and scoping. So there's a bit of rose-colored glasses toward the past here.

Second, the idea that time estimation is no longer applicable is probably off. There are two separate problems: problem definition, and problem solving, and estimation is extremely useful in the latter. Problem definition is a different problem that still needs much focus, but it doesn't preclude the need for better understanding of time to coordinate other processes and dependencies. You might be saying that those dependencies aren't as important as we think, and I tend to agree, but that's a different argument.

Generally speaking, the idea that knowledge and skills from the industrial era are no longer applicable is untrue. There is a huge body of knowledge about how products are made and built that have 99% applicability to software and technology in the post-industrial world. This is because the problems are the same: management of people, leadership, understanding interactions within complex systems, understanding statistics (the importance of which this article proves profoundly), and improving the spread of knowledge. This is the way that Toyota began operating in the post-WWII era, the way W. Edwards Deming modeled companies, and the way that the current Lean movement guides you to improve almost any business. It's highly relevant.

The main point we should take away, is that time and estimates are not constraints on a system; but rather outputs that are predictable and follow statistical patterns. We can use those outputs to make better decisions, especially if we understand the whole process of production in a systemic way.

Estimation need not die. It's a tool for good in the hands of a systems thinker.

Re: Why Are Projects Always Behind Schedule?

#54
post #37

The fact that planning estimates represent a probability distribution is well known and there is an established process (PERT)[1] for estimating the expected time of a set of estimates - essentially instead of asking for a single 'most likely' estimate you should as well explicitly walk through some of the risks around this and the ask for an 'optimistic' estimate (which is frequently very similar to the initial esti…

Is there any reason to use a triangular distribution these days? I can see it made sense in the 1950s, but now it should be trivial to use a more accurate distribution.

Re: Why Are Projects Always Behind Schedule?

#57
My son keeps telling me about some study that found that optimists fall further behind than pessimists, but it wasn't what they expected. They expected (as a made up example) that if it required 30 days, pessimists would estimate 40 and perhaps come in on time and optimists would estimate 20 and not make it. Reality was more like it would require 40 days and pessimists would estimate 30 and optimists would estimate 20. They both got it wrong, optimists just got it more wrong.

Another issue: It is really common for people to "take some well deserved time off!" when they finish some piece of the project earlier than anticipated, thus flushing away time that could have helped out on parts that will take longer than expected.

Cuz: Humans.

Re: Why Are Projects Always Behind Schedule?

#58

Earlier quoted context omitted.

According to this reasoning you should set targets based on expected time, not "median" time. You're saying the tail of "completion time" is long, and so the median is a poor estimator for expected time. But if you actually knew the distribution of times for task completion (which is implied if you know the 'median time'), you could simply use the expected value of this distribution and your projects would then tend…

Exactly. All this article is really saying is "most of the time, adding together median expected times underestimates the sum of mean expected times". Well then, easy fix: add together the mean expected times off the bat.

There's more to it than your easy fix, the article alludes to it but does a poor job of covering it: if you know the distributions of the intermediate steps, but you only sum the means of the steps, then you are throwing away data that could help you calculate a better estimate.

Re: Why Are Projects Always Behind Schedule?

#59
post #9

There is actually a nugget of thought in the article beyond the explanations you always hear (and which make up all the comments in this thread as I'm writing this): Even if you base your project schedules on accurate estimates of the median time required for each step, you will still be behind schedule for most projects. The reason is actually quite simple: when things go well, they can only go so well, but when the…

In addition to that, project's stages ahead-of-schedule deliveries are typically wasted, but behind-of-schedule deliveries are accumulated.

As an example: Arriving to the airport one hour in advance doesn't allow you to actually board a flight an hour earlier. But get to the airport one hour late — and you're flying tomorrow (or whenever the next flight is scheduled).

Re: Why Are Projects Always Behind Schedule?

#60
The time it will take to complete a project is fixed but unknowable. The only thing you can control is the scope and the cost for each unit of work. Therefore the only strategy is to do the least amount of work you can possibly do for the least possible cost. If someone requires you to fix your costs you have to charge them a lot more money. Best to tell them that whatever happens, nothing could have been done to improve it and make sure they're willing to compromise heavily to get it live.
Post reply on HN