What do you mean ‘we need more time’? Project schedule estimation in software

Unfortunately, that doesn't change the fact that customers want estimates, and the more complex the project the more they need it for budgeting and scheduling purposes.

In the end, you have a few options for estimating:

1. Estimate the most optimistic scenario, knowing you'll end up using change requests and extensions to deliver (one of the worst options, but if you're responding to a proposal that's what the opposition is assuredly going to be doing);

2. Propose an initial phase to nail down requirements and deliver a schedule using traditional estimation tools (an easier sell if you're an internal team, tough if you're a vendor);

3. Use experience with similar projects to estimate, and then refine your assumptions with the client, using change orders to account for unexpected problems (roughly accurate to the degree of imported ambiguity, but now you have to risk going to war to get your change orders signed off on);

4. Use a top-down estimation model such as function points-based estimation to get in the ballpark, and then add buffer (and pray you aren't going to either lose money or blow the top off the client's budget levels -- or both);

5. Delphi the problem and get a bunch of developers and PMs to estimate the effort, then refine from there (useful for sanity checks, not useful as a real estimation process, yet one of the more common ways small shops build their numbers).

As you say, you can't make risk go away, you can only try to segregate it within your estimation. In my experience, clients really need stability in proposal numbers -- they're often willing to accept a significant embedded risk buffer in a proposal if it means they can limit unexpected costs. Conversely, you should also have a rough idea going into a project of what functionality can be cut and when so you can go/nogo those pieces when necessary.

The problem isn't with the requirement for fixed-time, fixed-bid contracts. The problem is that agile methods and tools, and all the advantages they carry, are incompatible with those requirements: You can't start fast with minimal specs. You can't apply what's been learned along the way unless, miraculously, it costs less and takes less time. You can't make changes to suit changing business goals without a major negotiation on change orders. You can't access most of the benefits of agile inside that box.

You CAN, however, find contract developers who have mastered the art of faking agile methods and being buzzword compliant while delivering no feedback that prevents you from having bad ideas implemented in bad ways.

You will also find contractors that will lead you down the garden path. You could call them "half agile." They won't warn you that you are asking for something dumb, or unworkable. They will treat your mistakes as revenue enhancement opportunities and lay on the agile talk pretty thick.

This is why the "no true agile Scotsman" argument is so easy to make: That's not agile. You have the wrong people. You fail at agile. That's an easy case to prove but it isn't very helpful.

Real agile is hard to do and doing it with compromised ingredients, like using low-bid contractors that want to minimize their effort and/or enhance their revenue, that you have stuffed into a fixed-bid box, is agile poison.

Actually it's remarkably similar. Sales has a pipeline of deals, and someone has an estimate of how much will be booked over the next quarter. This estimate is a key part of managing the sales team and is directly tied to their compensation.

Can you imagine if developers explicitly had a code quota to work to? We usually don't, for very good reasons, but salespeople certainly are held to delivery standards on work that isn't fully predictable, so you can kinda see how they might expect the same from software teams.

And the analogy continues to work. There is a pipeline of feature requests, some at the vague-idea stage and some nearly shipped. You can provide more accurate ship-date estimates for the nearly shipped ones. And a person with good judgment who knows the stakeholders can provide a better estimate than someone uninformed or unthoughtful whether we're talking about sales or software builds.

Further: sales has a bookings target, but also has the freedom to meet that target with various deals. Similarly, product teams will have launch targets, but need some freedom on what exact features are included because per-feature costs aren't predictable.

Customers don't need to know the cost, they need to know the price. The most important factor in determining that should be the value of the software to the customer, not the time it will take to produce it.

As a new grad who has worked a couple different jobs now, some things I don't get about this:

- What do you do when you don't have a spec and when you ask the senior engineer you are working with questions, they don't respond or push back against having a spec or a plan longer than the very next feature?

- The author says that estimation is easier to do if you are doing something you have done before. What do you do if you are always working with a new framework or language?

- How do you break down the steps if you don't know what a project entails until you are finished?

- What do you do if asked to estimate how long something will take to debug? What if you just started on a project and the error message you are trying to debug doesn't make sense? My approach there is to try to understand the system I am working with. If the system doesn't have documentation and the existing people on the project don't have time to give a walk through of things, I figure that means source diving. However, I've gotten the feedback that I should avoid trying to "understand the universe" when debugging. What can I do to produce accurate estimates of how long it takes to find a bug in a system without spending the time to gain a mental model of the system?

- This excercise takes a lot of time. What do you do when asked for an estimate in a meeting rather than over email? What

- What do you do if the thing you are building relies on getting an external API to work and that API is either undocumented or is documented in a foreign language? Assume that the google translation is not making sense and this is your first project and you don't have a budget to hire a professional translator? This sounds like something you would need to get working before you could actually give an estimate.

Why do experienced engineers ask for estimates anyway? Every time I give one I feel like I am lying and I try to warn people "I really don't know how to come up with estimates but I am guessing 3 hours". How do I deal with it when they are upset about it taking a week and a half?