What do you mean ‘we need more time’? Project schedule estimation in software • Hacker News Board

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#71

Post by diego_moita » Fri, Feb 12, 2016, 4:26 PM UTC

Kurtz79 wrote: post #39

13hours wrote: post #7

Almost at the end : "Yes, all this sounds like a lot of work". Exactly. And it will most probably still turn out to be wrong, because you will forget to ask some questions, and there will be things that you simply cannot foresee that will influence the time. Turns out it's pointless to even try to do this. So why waste valuable time to do all this, if you know it's not the truth anyway? The answer is to accept that y…

"So why waste valuable time to do all this, if you know it's not the truth anyway ? The answer is to accept that you will learn things as you go along, and that things take as long as they take, and to rather deal with that." This maybe is applicable for a company developing new products internally, but for projects delivered to third parties, delivery dates are part of contractual obligations. Estimates are hard, bu…

> as they gain experience in the domain

Because "gain experience" means that there are no more "unknown unknowns" for them, anymore.

But, unfortunately that is a luxury you can't afford for too long. Reality has the nasty habit of changing, shit happens and good developers need to get replaced as they burn out and run away from development into more sane activities.

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#72

Post by jsprogrammer » Fri, Feb 12, 2016, 4:26 PM UTC

eterm wrote: post #21

13hours wrote: post #20

Earlier quoted context omitted.

I don't think working fixed price quote would have given you a different outcome though? The team produced bad quality product, fixed cost or agile won't change that. Selecting the team to build the product on price is often a bad idea, and that's all fixed cost gives you above agile (the ability to select on price).

A fixed cost makes it easier to turn and around not pay (in whole or part) if the product is really not up to the standard specified. It's harder to do that if the cost is not fixed but overrunning and the response is "well we just need more time". Also fwiw, the team wasn't selected on price.

Not pay? Heh. For work performed? OK...

That is why my contracts require payment up front.

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#73

Post by lmm » Fri, Feb 12, 2016, 4:28 PM UTC

TheLogothete wrote: post #70

lmm wrote: post #66

Earlier quoted context omitted.

> Before NASA sent their first drone to Mars they made a lot of predictions, a lot of tests, guessed a lot of things that may get wrong, etc. Do some of their missions still fail? Sure. But the amount of success is much much larger because they didn't just sent a drone there and watched what happened. How much effort went into estimating Opportunity's lifetime at 90 days? How useful has that estimate proven to be?

That is cherry picking. Did you try to comprehend the post you are replying to? Take into account all of the estimations, not just one failed estimation.

Sure, take into account all the estimations. How much value have they delivered? Show me how NASA has got more science done than if they had "just sent a drone there and watched what happened". Because I don't believe they have.

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#74

Post by diego_moita » Fri, Feb 12, 2016, 4:29 PM UTC

jsprogrammer wrote: post #68

Kurtz79 wrote: post #39

Earlier quoted context omitted.

"So why waste valuable time to do all this, if you know it's not the truth anyway ? The answer is to accept that you will learn things as you go along, and that things take as long as they take, and to rather deal with that." This maybe is applicable for a company developing new products internally, but for projects delivered to third parties, delivery dates are part of contractual obligations. Estimates are hard, bu…

Don't put hard commitments into the contracts then. Why slave yourself to something that you aren't sure you can deliver? Good developers should ideally only ever do the same thing once, which means that there is very limited utility in trying to improve estimates for a task that you should never perform again.

The secret of a consultant's work is not to craft software, it is to craft contracts.

Evidence : CGI never got burned by their fiasco on delivering Healthcare.gov

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#75

Post by gadders » Fri, Feb 12, 2016, 4:41 PM UTC

13hours wrote: post #7

Almost at the end : "Yes, all this sounds like a lot of work". Exactly. And it will most probably still turn out to be wrong, because you will forget to ask some questions, and there will be things that you simply cannot foresee that will influence the time. Turns out it's pointless to even try to do this. So why waste valuable time to do all this, if you know it's not the truth anyway? The answer is to accept that y…

"Hi. I need you to build me a house. It needs to have three bedrooms, two garages etc. How long will it take?"

"Well, we can definitely get you something by Christmas. It might be a bedroom, we may have just done the garages. Who knows? Can you write the cheque now please?"

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#76

Post by p4wnc6 » Fri, Feb 12, 2016, 4:49 PM UTC

beat wrote: post #69

The tendency to estimate poorly - particularly to underestimate - is one of the key reasons to use a very tight, short-iteration agile process. If I estimate something will take two days, and I'm off by 50%, it'll take three days. If I estimate it'll take two years, and I'm off by 50%, it'll take three years. And because I can get a lot more practice at two day schedules than two year schedules, I'll improve my accur…

I've never experienced estimation improving over time in an Agile process. In fact, since short estimation cycles are just as susceptible to political games as longer cycles, and this dominates the whole process in any organization, the length of the estimation cycle really has nothing to do with the estimation issue.

But, what shorter estimation cycles are responsible for is a decoupling of the overall big picture progress from the immediate tasks being prioritized. This often leads to sprints that are well scoped, and everything in the sprint gets completed close to on time (at least as close as in longer work cycles, but not usually any closer), but, crucially all of the work has to be scrapped because the whole sprint, conceptually, was not right. In my experience, this happens maybe 1/5 of the time in Agile exactly in ways that would be prevented with longer-term planning.

It's very similar to the classical fractal effects of measuring the coast of Britain. By refining your ruler, you merely think you're being more accurate, and a lot of people are doing a lot of performative managerial crap for the sake of the Scrum performance, like a ritual. It feels like you are estimating better, but really because the whole meta-level concept you are working on isn't guided by longer-term thinking, you end up having so much round off error, added up over more and more cycles, that the overall waste is colossally greater than systems that involve some more significant longer-term pre-planning and which work on variable cycle lengths instead of a cookie-cutter, one-size-fits-all approach (which is the core philosophy of Agile, though people try to obfuscate this fact by No True Scotsman-ing it with the Agile Principles).

I also can't miss the opportunity to point out this great article on the political games we play during estimation: http://research.cs.queensu.ca/home/ahmed/home/teaching/CISC3... >

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#77

Post by stronglikedan » Fri, Feb 12, 2016, 5:03 PM UTC

DougWebb wrote: post #57

jameshart wrote: post #48

Earlier quoted context omitted.

Customers don't need to know the cost, they need to know the price. The most important factor in determining that should be the value of the software to the customer, not the time it will take to produce it.

True, but difficult to sell, especially to repeat customers who can calculate the price/hr on previous work.

I have no problem telling a customer that something along the same lines as a previous project will be more expensive today, because it was initially underestimated and I ate the additional cost. They realize they got a bargain the first time, and I don't have to eat the same costs to keep the customer. Of course, starting from the first project, I work very hard on building a trustful relationship to support such a claim.

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#78

Post by SapphireSun » Fri, Feb 12, 2016, 5:05 PM UTC

jsprogrammer wrote: post #72

eterm wrote: post #21

Earlier quoted context omitted.

A fixed cost makes it easier to turn and around not pay (in whole or part) if the product is really not up to the standard specified. It's harder to do that if the cost is not fixed but overrunning and the response is "well we just need more time". Also fwiw, the team wasn't selected on price.

Not pay? Heh. For work performed? OK... That is why my contracts require payment up front.

Sounds like they should have negotiated for a base rate with bonus pay out on meeting QC targets. For heavens sake people, align your incentives!

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#79

Post by Gratsby » Fri, Feb 12, 2016, 5:15 PM UTC

I'm about to being a rant here because this is a larger problem than anybody likes to admit.

Developers are good at development. By and large, they are not good at task management, project management, and certainly not estimation. One of the primary reasons for this is that their plates are always overflowing and they work more efficiently than many people in other professions. They shift priorities on the fly when work in one area cannot be progressed. Development time is social currency. You can make your boss happy by progressing his priority, or you can make someone else happy by progressing their priorities. Not many developers truly answer to a singular master. And few allocate time for learning although we all spend a significant amount of time doing it - how else do we maintain relevant skill sets?

Can a developer really estimate a project and tell his manager that 3 weeks in the timeline are for 2 unknown and unforeseen problems that require interaction with vendor support? No, because the manager will fight back and say that those 3 weeks are really just zipping up logs and making three phone calls and "if it takes anything more than that, hand it over to me!" Most can't tell their manager that it will take 2 weeks to parse a CSV because the source data is shit and they have too many other things they are working on. So they stay an extra 6 hours in the office and get called a rock-star until the day that they opt to go to their child's school play instead.

The solution to this problem is simple. Take task management and project estimation out of the hands of developers. It's time for the return of the secretary to the workplace. A single $50-80k administrative assistant could easily support, manage the priorities of, and improve the outward facing communication for five developers. With experience doing so, that admin could provide better time estimates for all of them and would be better at negotiating realistic timelines than any of them individually.

People have been coming up with methodologies and software to solve this problem for decades. Developers don't have brains built for project management. Yes, they can do it, but they don't get paid to work in MS Project. They get paid because they know the difference between a bubble sort and a quick sort and when to use either. Conversely, I don't know a soccer mom who doesn't successfully manage a more challenging schedule and competing set of priorities better than every nose-down developer I've ever met. Give appropriate work to appropriate people. Developers have their talents. Scheduling, estimating, and task management are not among them.

Re: What do you mean ‘we need more time’? Project schedule estimation in software

#80

Post by beat » Fri, Feb 12, 2016, 5:19 PM UTC

p4wnc6 wrote: post #76

beat wrote: post #69

The tendency to estimate poorly - particularly to underestimate - is one of the key reasons to use a very tight, short-iteration agile process. If I estimate something will take two days, and I'm off by 50%, it'll take three days. If I estimate it'll take two years, and I'm off by 50%, it'll take three years. And because I can get a lot more practice at two day schedules than two year schedules, I'll improve my accur…

I've never experienced estimation improving over time in an Agile process. In fact, since short estimation cycles are just as susceptible to political games as longer cycles, and this dominates the whole process in any organization, the length of the estimation cycle really has nothing to do with the estimation issue. But, what shorter estimation cycles are responsible for is a decoupling of the overall big picture p…

All excellent points. But having done big waterfall, bad corporate agile, and good agile, the problem of work failing conceptually is common to all of them. The question is, how do you recover from it? I think good agile recovers more gracefully from conceptual failures than either waterfall (which tends to throw good money after bad, because admitting failure is not an option), or corporate agile (because corporate agile is usually just buzzword-compliant waterfall).